search

xinntao / Real-ESRGAN-ncnn-vulkan

NCNN implementation of Real-ESRGAN. Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration.
https://github.com/xinntao/Real-ESRGAN
Other
1.44k stars 173 forks source link

Issue with audio sync #10

Open w2lf opened 2 years ago

w2lf commented 2 years ago

I am trying to enhance anime video but the audio does not match the original video. Original video is 1:40 and out video is 1:39, tried -r 24. Same problem. has anyone else noticed this? The audio doesn't sync with the original video. Frames are the same in the tmp_frames folder and out_frames folder. Same happens if I do jpg and at x2 scale. So I think its something with the ffmpeg.

Link to the out video. If you go to description and find the original video. You can then see the audio not sync right. https://www.youtube.com/watch?v=qF2pXR6daUU

Using this commands as in readme, does anyone else run into this problem? Commands for enhancing anime videos:

  1. Use ffmpeg to extract frames from a video (Remember to create the folder tmp_frames ahead) ffmpeg -i onepiece_demo.mp4 -qscale:v 1 -qmin 1 -qmax 1 -vsync 0 tmp_frames/frame%08d.png

  2. Inference with Real-ESRGAN executable file (Remember to create the folder out_frames ahead) ./realesrgan-ncnn-vulkan.exe -i tmp_frames -o out_frames -n realesr-animevideov3 -s 4 -f png

  3. Merge the enhanced frames back into a video ffmpeg -i out_frames/frame%08d.png -i onepiece_demo.mp4 -map 0:v:0 -map 1:a:0 -c:a copy -c:v libx264 -r 23.98 -pix_fmt yuv420p output_w_audio.mp4

NaokiSato102 commented 2 years ago

It seems to me that this case is more about your proficiency with ffmpeg than our application.

M4he8hw4r4 commented 1 year ago

Check through ffprobe the number of frames in the video after the build, with "-r 23.98" I got less than the original video. The problem was solved by replacing "-r 23.98" with "-framerate 23.98"

Dark25 commented 1 year ago

Check through ffprobe the number of frames in the video after the build, with "-r 23.98" I got less than the original video. The problem was solved by replacing "-r 23.98" with "-framerate 23.98"

Yes, but at the beginning you must place it

ashsii commented 7 months ago

For anyone in the future trying to solve this issue. "FFmpeg is dropping frames because the input framerate doesn't match the output framerate". Try this instead it fixed it for me:

ffmpeg -f image2 -framerate 23.98/1 -i out_frames/frame%08d.jpg -i original_video.mp4 -map 0:v:0 -map 1:a:0 -c:a copy -c:v libx264 -r 23.98 output_w_audio.mkv

Also add -map 1:s:0 and change mp4 to mkv to transfer subtitles.

mlbrothers commented 6 months ago

Not exactly related but how do I use model for video?