Audio Codec options or changing the default.

[lolcatplays]

When uploading clips to twitter which is what I primarily use this for, the audio codec that is chosen by default doesn't work. I think I only got it to work after re encoding the whole video which ruined quality further. Reddit works fine but Twitter is my main form of social media.

[Smalls1652]

Noticed that too, but on iOS and macOS (If you use quick view from Finder or QuickTime) the audio stream is not playing. Ran ffprobe on one of the files and it came back with h264 on the video stream and mp3 on the audio stream.

  Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x800 [SAR 1:1 DAR 8:5], 889 kb/s, 30.01 fps, 30 tbr, 16k tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : "VA-API\ H264\ encoder"
  Stream #0:1[0x2](eng): Audio: mp3 (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
[h264 @ 0x15a00dc60] time_scale/num_units_in_tick invalid or unsupported (0/1)

Not sure why they're not playing the audio stream though. IIRC mp3 codecs are supported in the MP4 container format.

I forked the repo and made a change to the ffmpeg process that runs after saving a rolling/replay recording video that copies the existing video stream and encodes the audio stream directly to aac at 128kbps. That'll prevent the video from being re-encoded and degrading the quality any further; however, the same can't be said for the audio and there might be some loss in quality.

Smalls1652/decky-recorder-fork/blob/61954ae3d013ab3f69e2f5484feb97ff8c2cc3e9/main.py#L362

I built it locally and deployed it to my Deck and it's working. If you want me to @safijari, I can open a PR for it. The other thing we could possibly do is change the audio encoder in the GStreamer pipeline to something that encodes to aac directly from the raw audio (faac maybe?). That would probably be the much better option than remuxing a mp3 to aac, since you're not encoding twice (raw -> mp3 -> aac) and degrading the quality of the audio. I've got a second branch on my fork with faac implemented. Just waiting for the build to finish so I can deploy it to my Deck and test it.

Edit:

Mmm I see why faac was replaced. If that audio delay is still there, the first option would probably be the only way to get the MP4 container to work properly.

[safijari]

Yes please open the PR and I'll review it. Someone included faac a while back but I found that with it the recordings kept getting desynced. We can try it again.

On Mon, Dec 18, 2023, 8:19 PM Timothy Small @.***> wrote:

Noticed that too, but on iOS and macOS (If you use quick view from Finder or QuickTime) the audio stream is not playing. Ran ffprobe on one of the files and it came back with h264 on the video stream and mp3 on the audio stream.

Stream #0:00x1: Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x800 [SAR 1:1 DAR 8:5], 889 kb/s, 30.01 fps, 30 tbr, 16k tbn (default) Metadata: handler_name : VideoHandler vendor_id : [0][0][0][0] encoder : "VA-API\ H264\ encoder" Stream #0:10x2: Audio: mp3 (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default) Metadata: handler_name : SoundHandler vendor_id : [0][0][0][0] [h264 @ 0x15a00dc60] time_scale/num_units_in_tick invalid or unsupported (0/1)

Not sure why they're not playing the audio stream though. IIRC mp3 codecs are supported in the MP4 container format.

I forked the repo and made a change to the ffmpeg process that runs after saving a rolling/replay recording video that copies the existing video stream and encodes the audio stream directly to aac at 128kbps. That'll prevent the video from being re-encoded and degrading the quality any further; however, the same can't be said for the audio and there might be some loss in quality.

Smalls1652/decky-recorder-fork/blob/61954ae3d013ab3f69e2f5484feb97ff8c2cc3e9/main.py#L362 https://github.com/Smalls1652/decky-recorder-fork/blob/61954ae3d013ab3f69e2f5484feb97ff8c2cc3e9/main.py#L362

I built it locally and deployed it to my Deck and it's working. If you want me to @safijari https://github.com/safijari, I can open a PR for it. The other thing we could possibly do is change the audio encoder in the GStreamer pipeline to something that encodes to aac directly from the raw audio (faac maybe?). That would probably be the much better option than remuxing a mp3 to aac, since you're not encoding twice (raw -> mp3 -> aac) and degrading the quality of the audio. I've got a second branch https://github.com/Smalls1652/decky-recorder-fork/tree/audio-codec-fix-option-02 on my fork with faac implemented. Just waiting for the build to finish so I can deploy it to my Deck and test it.

— Reply to this email directly, view it on GitHub https://github.com/safijari/decky-recorder-fork/issues/17#issuecomment-1861996916, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABHTRJAHEUQWTE67BOOR4NDYKD2TTAVCNFSM6AAAAAA72MZ5BCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRRHE4TMOJRGY . You are receiving this because you were mentioned.Message ID: @.***>

[Smalls1652]

I unfortunately tested a more interactive recording and it doesn't exactly solve the issue. It's pointing towards the timescale being off in the video stream (#18). If I re-encode the video on my laptop (The audio stream is already encoded with aac) with ffmpeg -i xyz.mp4 -c:v libx264 -c:a copy xyz_converted.mp4, the video and audio streams are in sync.

I'm digging around to see if I can figure out why the VAAPI H264 encoder is doing that.

[Smalls1652]

So I've got a bit of a band-aid fix. I've got ffmpeg set to resample the audio stream (with -af aresample=async=1000) and that's resolving the audio stream being out of sync with the video stream. Here's a sample video. It might be distorting the audio a little bit? I also re-enabled faac in the GStreamer pipeline and bumped the bitrate to 320kbps. It's not an ideal solution because we're still re-encoding the audio later on with ffmpeg at 192kbps. Not a huge fan of lossy-to-lossy conversion. 😅 Could also be an issue for non-rolling recording videos?

I'm still trying to figure out what might be the best solution to fix the timescale, but this should get usable MP4 files. I'll create the PR for it tomorrow and I can make any changes if needed.