search

bytedeco / javacv

Java interface to OpenCV, FFmpeg, and more
Other
7.45k stars 1.57k forks source link

when I stream my video for about 2 min I got an error "av_interleaved_write_frame() error -32 while writing interleaved video packet" #2023

Open sunflower-zzn opened 1 year ago

sunflower-zzn commented 1 year ago 
 public void liveStreamingVideos(List<String> urls, String rtmpAddress, String tenantCode, int maxSize, double frameRate) {
        try {
            avutil.av_log_set_level(avutil.AV_LOG_INFO);
            FFmpegLogCallback.set();
            int count = 0;
            FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(rtmpAddress, 800, 450, 1);
            recorder.setFrameRate(25); 
            recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264); 
            recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC); 
            recorder.setFormat("flv");
            long time = 0;
            for (String url : urls) {
                FFmpegFrameGrabber grabber = new FFmpegFrameGrabber("/Users/Downloads/1.webm");
                grabber.setOption("analyzeduration", "10000");
                grabber.setFormat("webm");
                grabber.setFrameRate(25);
                grabber.start();

                int frameWidth = grabber.getImageWidth();
                int frameHeight = grabber.getImageHeight();

                if (count == 0) {
                    recorder.start();
                    count++;
                }

                grabber.flush();
                Frame pkt = null;
                while ((pkt = grabber.grabAtFrameRate()) != null) {
                    try {
                       recorder.setTimestamp(pkt.timestamp + time);
                        time = pkt.timestamp + time;

                        recorder.record(pkt);
                        LoggerUtil.info(LOGGER, "push " + pkt.pictType + " : " + time);
                        Thread.sleep((long) frameRate);
                    } catch (Exception e) {
                       System.out.println("jump" +  pkt.pictType + e);
                    }
                }
                grabber.close();
            }
            recorder.close();
        } catch (Exception e) {
            LoggerUtil.error(LOGGER,  e);
        }
}
saudet commented 1 year ago

That sounds like an issue with the network. Check the log for more information, and for more reliability, make sure you're using TCP and not UDP as protocol.

saudet commented 1 year ago

Not the log of your application, FFmpeg's log. Please make sure that FFmpegLogCallback.set() has been called.

sunflower-zzn commented 1 year ago

Here is all FFmpeg's log and exception av_interleaved_write_frame() error -32 while writing interleaved video packet

saudet commented 1 year ago

Maybe libopenh264 is causing problems. Try another codec like libx264 instead.

sunflower-zzn commented 1 year ago

how to use libx264?

saudet commented 1 year ago

Please add a dependency on ffmpeg-platform-gpl as shown in the README.md file here: https://github.com/bytedeco/javacv#sample-usage

saudet commented 1 year ago

And my another question is how to use TCP as protocol ? Add: grabber.setOption("rtsp_transport", "tcp"); ?

Yes, that typically works for that purpose.

sunflower-zzn commented 1 year ago

I still have the problem av_interleaved_write_frame() error -32

Debug: [matroska,webm @ 0x7ff4eab72ac0] Opening '/Users/caoshuhua/Downloads/1.webm' for reading

Debug: [file @ 0x7ff4f9a7c6c0] Setting default whitelist 'file,crypto,data'

Debug: st:0 removing common factor 1000000 from timebase

Debug: st:1 removing common factor 1000000 from timebase

Debug: [matroska,webm @ 0x7ff4eab72ac0] Before avformat_find_stream_info() pos: 978 bytes read:32768 seeks:0 nb_streams:2

Debug: [opus @ 0x7ff4eab73780] skip 312/960 samples

Debug: [vp9 @ 0x7ff4eab73080] Format yuv420p chosen by get_format().

Debug: [matroska,webm @ 0x7ff4eab72ac0] All info found

Debug: [matroska,webm @ 0x7ff4eab72ac0] After avformat_find_stream_info() pos: 8251 bytes read:32768 seeks:0 frames:2

Info: Input #0, matroska,webm, from '/Users/caoshuhua/Downloads/1.webm':

Info:   Metadata:

Info:     COMPATIBLE_BRANDS: 
Info: qt  
Info: 

Info:     MAJOR_BRAND     : 
Info: qt  
Info: 

Info:     MINOR_VERSION   : 
Info: 512
Info: 

Info:     ENCODER         : 
Info: Lavf58.76.100
Info: 

Info:   Duration: 
Info: 00:00:24.48
Info: , start: 
Info: -0.007000
Info: , bitrate: 
Info: 145 kb/s
Info: 

Info:   Stream #0:0
Debug: , 1, 1/1000
Info: : Video: vp9 (Profile 0), 1 reference frame, yuv420p(tv, bt470bg/unknown/unknown, progressive, left), 800x450, 0/1
Info: , SAR 1:1 DAR 16:9
Info: , 
Info: 25 fps, 
Info: 25 tbr, 
Info: 1k tbn
Info:  (default)
Info: 

Info:     Metadata:

Info:       HANDLER_NAME    : 
Info: VideoHandler
Info: 

Info:       VENDOR_ID       : 
Info: FFMP
Info: 

Info:       ENCODER         : 
Info: Lavc58.134.100 libvpx-vp9
Info: 

Info:       DURATION        : 
Info: 00:00:24.447000000
Info: 

Info:   Stream #0:1
Debug: , 1, 1/1000
Info: : Audio: opus, 48000 Hz, mono, fltp, delay 312
Info:  (default)
Info: 

Info:     Metadata:

Info:       HANDLER_NAME    : 
Info: SoundHandler
Info: 

Info:       VENDOR_ID       : 
Info: [0][0][0][0]
Info: 

Info:       ENCODER         : 
Info: Lavc58.134.100 libopus
Info: 

Info:       DURATION        : 
Info: 00:00:24.480000000
Info: 

Debug: detected 12 logical cores

Debug: [flv @ 0x7ff4eae54a00] intra_quant_bias = 0 inter_quant_bias = -64

Debug: [rtmp @ 0x7ff4fded4a00] No default whitelist set

Debug: [tcp @ 0x7ff4f9ab1200] No default whitelist set

Debug: [tcp @ 0x7ff4f9ab1200] Original list of addresses:

Debug: [tcp @ 0x7ff4f9ab1200] Address 203.209.245.155 port 1935

Debug: [tcp @ 0x7ff4f9ab1200] Address 203.209.250.124 port 1935

Debug: [tcp @ 0x7ff4f9ab1200] Interleaved list of addresses:

Debug: [tcp @ 0x7ff4f9ab1200] Address 203.209.245.155 port 1935

Debug: [tcp @ 0x7ff4f9ab1200] Address 203.209.250.124 port 1935

Debug: [tcp @ 0x7ff4f9ab1200] Starting connection attempt to 203.209.245.155 port 1935

Debug: [tcp @ 0x7ff4f9ab1200] Successfully connected to 203.209.245.155 port 1935

Debug: [rtmp @ 0x7ff4fded4a00] Handshaking...

Debug: [rtmp @ 0x7ff4fded4a00] Type answer 3

Debug: [rtmp @ 0x7ff4fded4a00] Server version 13.14.10.13

Debug: [rtmp @ 0x7ff4fded4a00] Proto = rtmp, path = /live/A202305092247540101007399?auth_key=1748429772-0-0-eecd79f09c5e162c90acbe740bec5613, app = live, fname = A202305092247540101007399?auth_key=1748429772-0-0-eecd79f09c5e162c90acbe740bec5613

Debug: [rtmp @ 0x7ff4fded4a00] Window acknowledgement size = 5000000

Debug: [rtmp @ 0x7ff4fded4a00] Max sent, unacked = 5000000

Debug: [rtmp @ 0x7ff4fded4a00] New incoming chunk size = 4000

Debug: [rtmp @ 0x7ff4fded4a00] Releasing stream...

Debug: [rtmp @ 0x7ff4fded4a00] FCPublish stream...

Debug: [rtmp @ 0x7ff4fded4a00] Creating stream...

Debug: [rtmp @ 0x7ff4fded4a00] Sending publish command for 'A202305092247540101007399?auth_key=1748429772-0-0-eecd79f09c5e162c90acbe740bec5613'

Info: Output #0, flv, to 'rtmp://live1.push.alipay.com.cn/live/A202305092247540101007399?auth_key=1748429772-0-0-eecd79f09c5e162c90acbe740bec5613':

Info:   Metadata:

Info:     encoder         : 
Info: Lavf59.27.100
Info: 

Info:   Stream #0:0
Debug: , 0, 1/1000
Info: : Video: flv1, 1 reference frame ([2][0][0][0] / 0x0002), yuv420p, 800x450 (0x0), 0/1, q=2-31, 400 kb/s
Info: , 
Info: 25 fps, 
Info: 1k tbn
Info: 

Info:   Stream #0:1
Debug: , 0, 1/1000
Info: : Audio: aac (LC) ([10][0][0][0] / 0x000A), 44100 Hz, mono, fltp, delay 1024, 64 kb/s
Info: 

Warning: [opus @ 0x7ff4da5185c0] Could not update timestamps for skipped samples.

Debug: [opus @ 0x7ff4da5185c0] skip 312/960 samples

Debug: [SWR @ 0x7ff4d9fd7000] Using fltp internally between filters

Debug: [vp9 @ 0x7ff4fd9ed740] Format yuv420p chosen by get_format().

Debug: [SWR @ 0x7ff4da350000] Using fltp internally between filters
2023-05-09 19:44:47,756 ERROR AVATAR-MNG - [GlobalExceptionHandler.handleException(62)][1ee723681683632661339102316361] --> global throwable exception.
org.bytedeco.javacv.FFmpegFrameRecorder$Exception: av_interleaved_write_frame() error -32 while writing interleaved video packet. (For more details, make sure FFmpegLogCallback.set() has been called.)
    at org.bytedeco.javacv.FFmpegFrameRecorder.writePacket(FFmpegFrameRecorder.java:1337)
    at org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(FFmpegFrameRecorder.java:1093)
    at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:971)
    at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:964)
    at com.alipay.avatarmng.core.service.media.impl.LiveStreamServiceImpl.liveStreamingVideos(LiveStreamServiceImpl.java:79)
    at com.alipay.avatarmng.web.test.test(test.java:30)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:681)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at com.alipay.fc.fcbuservice.sdk.BuserviceAccessControlFilter.doFilter(BuserviceAccessControlFilter.java:155)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at com.alipay.avatarmng.web.aop.ControllerFilter.doFilter(ControllerFilter.java:84)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at com.alipay.guardian.client.sofalite.WebFilterInvocation.proceed(WebFilterInvocation.java:99)
    at com.alipay.guardian.client.sofalite.GuardianFilterWrapper.invoke(GuardianFilterWrapper.java:53)
    at com.alipay.guardian.client.spring.GuardianMvcFilter.doFilter(GuardianMvcFilter.java:82)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at com.alipay.security.core.redirect.RedirectCheckFilter.doFilter(RedirectCheckFilter.java:51)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at com.alipay.security.core.securitymultipart.FileUpdateCheckFilter.doFilter(FileUpdateCheckFilter.java:83)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at com.alipay.security.core.defaultcors.DefaultCorsFilter.doFilter(DefaultCorsFilter.java:131)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at com.alipay.sofa.web.load.AlipayLoadTestFilter.doFilterInternal(AlipayLoadTestFilter.java:52)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at com.alipay.sofa.web.sofarouter.AlipaySofaRouterFilter.doFilterInternal(AlipaySofaRouterFilter.java:61)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at com.alipay.sofa.web.tracer.AlipaySofaTracerLogFilter.doFilterInternal(AlipaySofaTracerLogFilter.java:85)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at com.alipay.sofa.web.mdc.AlipayMDCInitFilter.doFilterInternal(AlipayMDCInitFilter.java:47)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at com.alipay.sofa.web.timer.AlipayTimerLogFilter.doFilterInternal(AlipayTimerLogFilter.java:69)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at com.alipay.sofa.web.metric.AlipayMetricFilter.doFilterInternal(AlipayMetricFilter.java:99)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:890)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1743)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
    at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

Info: [aac @ 0x7ff4f99db9c0] Qavg: 9332.791

Warning: [aac @ 0x7ff4f99db9c0] 2 frames left in the queue on closing

Debug: [AVIOContext @ 0x7ff4fda0ff80] Statistics: 1609577 bytes written, 0 seeks, 1510 writeouts

Debug: [rtmp @ 0x7ff4fded4a00] UnPublishing stream...

Debug: [rtmp @ 0x7ff4fded4a00] Deleting stream...

Debug: [AVIOContext @ 0x7ff4f99d9680] Statistics: 446256 bytes read, 0 seeks
sunflower-zzn commented 1 year ago

What else can I do to avoid the error?

sunflower-zzn commented 1 year ago

I found that the av_interleaved_write_frame() return 32 means Broken pipe, Does that mean I had livestreamed too fast or there's a problem with the bitrate?

sunflower-zzn commented 1 year ago

I fix the problem recently!!!! av_interleaved_write_frame() return 32 means Broken pipe, because of the video frame rate and audio frame rate are not stable, and in the process of livestreaming the video frame's timestamp always smaller than audio timestamp. so it need timestamp alignment then recoder.recode().

saudet commented 1 year ago

Call recorder.setTimestamp() if you have variable frame rate.

saudet commented 1 year ago 
                       recorder.setTimestamp(pkt.timestamp + time);
                        time = pkt.timestamp + time;

That doesn't look right. What happens if you call just recorder.setTimestamp(pkt.timestamp)?

saudet commented 1 year ago

You should also try to remove the call to Thread.sleep().

shuaisong commented 11 months ago

@sunflower-zzn Did you fix it?