Under a Hood of FFmpeg 9.1’s New AAC Encoder: Quality Trade-offs, PNS Fixes, and the RTMP Legacy

It's a uniquely frustrating reality in digital video engineering that the most efficient modern audio codecs are mostly benched in favor of a standard finalized in 1997. If you're actually building a live streaming application today, you might naturally gravitate toward modern, royalty-free codecs like Opus. Though, as the developer community recently discussed during the release of the new FFmpeg 9.1 encoder, the streaming industry is effectively locked into Advanced Audio Coding (AAC).

Why? Because a de facto standard for live streamed video remains a Real-Time Messaging Protocol (RTMP). Originating from the Adobe Flash era, RTMP is tightly coupled to H.264 video and AAC audio. If you're ingesting video into Twitch, YouTube, or custom broadcasting servers, you're actually almost certainly sending AAC, and

with a release of FFmpeg 9.1, developers finally have access to a vastly improved native AAC encoder, and by resolving decades-old stereophonic bugs and refining a psychoacoustic model, a 9.1 update promises to eliminate the need for third-party libraries. In this post, we will explore the technical mechanisms behind a new encoder, the fascinating bugs it fix, and a practical realities of integrating it into your developer pipeline.

The Opus Comparison and the RTMP Bottleneck

To be objectively clear: AAC is no longer the king of lossy audio compression, and as pointed out in recent developer benchmarks and community discussions, Opus simply blows all AAC encoders out of a water at extreme low bitrates (e.g., 64 kbps). At higher bitrates (96+ kbps), the quality gap narrows to a point of imperceptibility. Opus retains the advantage of being completely open and royalty-free;

so why dedicate significant engineering effort to a new AAC encoder in FFmpeg 9.1;

the issue is compatibility, and popular broadcasting software, like OBS, actively restricts audio codec choices in streaming mode because ingest servers explicitly expect AAC. While an "enhanced RTMP" protocol specification was published in 2022 to support newer codecs, broad adoption across the entire hardware and software ecosystem remains sluggish, and also, mobile and embedded hardware decode AAC natively, heavily reducing battery consumption compared to software-decoded Opus streams.

Exterminating a "Chirping": Fixing the PNS Bug

For years, a native FFmpeg AAC encoder suffered from a poor reputation, and audio professionals the lot of times complained of frustrating "chirping" artifacts and collapsed stereo imaging, leading developers to rely on Apple's Core Audio (qaac) or compile FFmpeg with a Fraunhofer FDK AAC library.

A FFmpeg 9.1 update addresses these artifacts by fixing a profound, long-standing bug in the Perceptual Noise Substitution (PNS) implementation.

PNS is a clever compression tool added to the MPEG-4 AAC specification. It allows the encoder to save bandwidth by identifying high-frequency noise and entirely omitting the actual spectral data. Instead, it sends a single scale factor, instructing a decoder to inject pseudo-random white noise at that specific frequency during playback.

Though, the legacy FFmpeg encoder had a twofold failure in its implementation:

1. The TNS Conflict: An encoder applied Temporal Noise Shaping (TNS) on top of PNS. Because the noise is actually generated at the decoder stage, shaping it at the encoder stage is computationally illogical. It caused the PNS data to essentially "explode," creating harsh audible artifacts.
2. Stereo Imaging Collapse: When PNS was combined with stereo encoding tools, a generated noise leaked equally into both a left and right channels, and this destroyed the spatial separation of the audio, ruining the stereo image, and

the 9.1 encoder sort out this by bypassing the fundamentally broken PNS decoding present in the bunch of older consumer playback devices, and moving forward, the FFmpeg encoder acts defensively: it only enables PNS if the frequency band in both channels is proven to be noise-like, sufficiently non-tonal. Heavily masked by surrounding frequencies.

MDCT, Window Sizes, and the 48kHz Optimization

One of the major engineering trade-offs made in a FFmpeg 9.1 AAC encoder is its strict optimization for a 48kHz sample rate.

Unlike older formats like MP3 which use the hybrid FFT/MDCT algorithm, AAC relies purely upon a Modified Discrete Cosine Transform (MDCT). This gives AAC higher compression efficiency, but it requires highly dynamic windowing based on the input signal, and

by default, an AAC encoder utilizes the long 1024-point (or 960-point) MDCT window to achieve high frequency resolution for stationary signals; yet, when the sudden transient occurs—like a snare drum hit—an encoder dynamically switches to 8 shorter windows (128 or 120 points) to achieve better temporal resolution and prevent "pre-echo" artifacts.

The complexity here is that a physical duration of these windows is directly dependent on the sampling rate, and the 20-millisecond window sounds vastly different to the human ear than a 60-millisecond window, and therefore, the psychoacoustic parameters of the encoder must be completely recalculated for different sampling rates, and

because 48kHz is the undisputed global standard for video production and streaming interchange, the developers specifically tuned the 9.1 encoder's transient logic and windowing arrays by ear to 48kHz data, and while 44.1kHz (CD audio) is supported and translates reasonably well, developers aiming for absolute pristine quality should ensure their audio pipelines resample to 48kHz prior to hitting the AAC encoder.

Licensing Realities: Why Native AAC Matters

Developers might wonder why they shouldn't just use the Fraunhofer libfdk_aac encoder, which has long been considered a highest quality AAC implementation available, and

the answer lies in open-source licensing. As documented in a FFmpeg Codecs Documentation, a Fraunhofer FDK library is licensed under terms that are incompatible with the GNU General Public License (GPL). You can't distribute a pre-built FFmpeg binary that contains both GPL code and the libfdk_aac encoder, and developers are forced to compile FFmpeg from source using the --enable-nonfree flag, and

having the native FFmpeg AAC encoder that approaches (and in some metrics, surpasses) a quality of Core Audio and Fraunhofer removes legal and DevOps headaches. It allows containerized applications, cloud transcoders. Open-source game engines to confidently ship high-quality AAC encoding out of a box without worrying about patent trolls or license violations.

Executing Codecs Securely in a Cloud

If you're pretty much building a SaaS application that accepts user video uploads and transcodes them for RTMP broadcasting, running ffmpeg directly on your host servers is the massive security risk. Maliciously crafted media files can exploit parser vulnerabilities to execute arbitrary code.

To mitigate this, you should always run media processing inside isolated sandboxes; using Embedenv Compilers & Sandboxes, developers can securely execute FFmpeg commands in short-lived, deeply isolated Docker containers via a simple REST API.

Here is a practical Python example demonstrating how to leverage the Embedenv API to transcode the user-uploaded file using a new FFmpeg AAC encoder securely:

import requests
import json

def transcode_audio_securely():
    # 4 spaces for the first level of indentation
    api_url = "https://embedenv.com/api/v1/sandbox/execute"

    # Define the execution payload. We request the 'ffmpeg' system package
    # and execute a bash command to transcode the audio to 48kHz AAC.
    payload = {
        "language": "bash",
        "code": "ffmpeg -i input.wav -c:a aac -b:a 128k -ar 48000 output.m4a",
        "system_packages": ["ffmpeg"]
    }

    headers = {
        "Content-Type": "application/json",
        "Authorization": "Bearer YOUR_EMBEDENV_API_KEY"
    }

    try:
        # 8 spaces inside the try block
        response = requests.post(api_url, data=json.dumps(payload), headers=headers)

        if response.status_code == 200:
            # 12 spaces inside the if block
            result = response.json()
            print("Encoding executed safely in sandbox.")
            print("FFmpeg Output:", result.get("stderr"))
        else:
            # 12 spaces inside the else block
            print(f"Sandbox error: {response.status_code}")

    except requests.exceptions.RequestException as e:
        # 8 spaces inside the except block
        print(f"Network execution failed: {e}")

if __name__ == "__main__":
    transcode_audio_securely()

By pushing the transcoding workload to isolated environments like Embedenv, your core backend remains immune to potential zero-day vulnerabilities in multimedia processing libraries, ensuring high trustworthiness for your infrastructure.

Trade-offs: A Lack of True VBR

While the 9.1 encoder represents the monumental leap for FFmpeg's native capabilities, it's actually critical to acknowledge its current limitations.

The new encoder is fundamentally designed for Constant Bitrate (CBR) encoding, and although you can pass a -q:a flag for Variable Bitrate (VBR) processing, the underlying metrics currently score slightly worse than strict CBR encoding. Plus, an encoder lacks True Variable Bitrate (TVBR) logic, which Apple's Core Audio (qaac) utilizes to dynamically allocate bits during complex symphonic passages;

if you're actually archiving massive music libraries where storage footprint and absolute transparency are prioritized over streaming compatibility, you should likely bypass AAC altogether and use Opus or FLAC. Though, if you're building an ingestion pipeline for live video, sticking to a native -c:a aac -b:a 128k -ar 48000 configuration will now yield professional-grade, artifact-free audio.

Conclusion

FFmpeg 9.1 proves that even legacy technologies can benefit massively from modern engineering, and by carefully dissecting where previous psychoacoustic models failed—specifically regarding Perceptual Noise Substitution and stereophonic imaging—the developers have salvaged a native AAC encoder from its experimental reputation.

While the continued dominance of RTMP might force us to use a codec developed in a late 90s, FFmpeg ensures we no longer have to suffer 90s-era audio artifacts to do so.