search

ossrs / oryx

Oryx(SRS Stack) is an all-in-one, out-of-the-box, and open-source video solution for creating online video services, including live streaming and WebRTC, on the cloud or through self-hosting.
https://ossrs.io/oryx
MIT License
519 stars 111 forks source link

AI Transcript acceleration by FFmpeg GPU support #159

Closed limjoe closed 6 months ago

limjoe commented 7 months ago

What is the business background? Please provide a description. AI transcript task is slow, especially ffmpeg encode part, so the transcript live stream with subtitle delay a lot.

Is your feature request related to a problem? Please describe. Large delay and poor user experience

Describe the solution you'd like AI transcript support GPU for ffmpeg encode, for example NVIDIA GPU for accelerate.

Describe alternatives you've considered

  1. Use a better CPU
  2. Use low-resolution video streaming

Additional context image

fonty37 commented 6 months ago 
ffmpeg \
    -hwaccel cuda -i input.ts -vf "subtitles=subtitles.srt:force_style='Alignment=2,MarginV=20',hwupload_cuda" \
    -c:v h264_nvenc -profile:v main -preset medium -rc:v vbr -cq:v 19 -bf 0 \
    -acodec aac \
    -copyts 
    -y output.ts
winlinvip commented 6 months ago

You are now able to setup the video codec parameters for transcript.

Image

Strongly recommand setting the -bt 0 to be compatible with both live stream and WebRTC.

winlinvip commented 6 months ago

Fixed in v5.14.11