AlexxIT / go2rtc

Ultimate camera streaming application with support RTSP, RTMP, HTTP-FLV, WebRTC, MSE, HLS, MP4, MJPEG, HomeKit, FFmpeg, etc.
https://github.com/AlexxIT/Blog
MIT License
4.32k stars 356 forks source link

Converting wonky MJPEG stream to RTC #1359

Open WayneManion opened 4 days ago

WayneManion commented 4 days ago

I have a Bambu P1S 3D printer. I have been trying to use an application called BambuP1Streamer to convert the video feed that comes out of the printer in some kind of MJPEG-over-MQTT format. BambuP1Streamer then uses go2rtc to stream "regular" MJPEG to the next consumer in the chain.

BambuP1Streamer works and I can see the MJPEG stream coming out of the go2rtc instance that is paired with BambuP1Streamer. I'm trying to take that MJPEG feed and convert it to RTC with another go2rtc instance so I can use it in Frigate and Home Assistant.

Sometimes this RTC stream does not work at all. Sometimes the RTC stream shows a frame, then the screen turns grey for a split-second, and another frame is shown. Sometimes the RTC stream works as well as the crappy camera in the printer can muster (1-2 fps).

I don't think it is a CPU or memory issue because this is running on a Xeon E5-2697v5 (18 cores with HT, 2.3 Ghz base clock) and 128 GB RAM. If I had to guess what was going on, its that the MJPEG stream that BambuP1Streamer and go2rtc are producing is just too uneven and choppy for ffmpeg to convert into RTC reliably. Is there some way to "frame double" the MJPEG stream or something to make it easier to convert to RTC?

I don't convert the BambuP1Streamer's feed to RTC in the first go2rtc instance because ffmpeg is not available in the container that BambuP1Streamer runs in.

First go2rtc stream config (BambuP1Streamer creating the MJPEG stream):

streams:
   p1s: "exec:./BambuP1Streamer ./libBambuSource.so ${PRINTER_ADDRESS} ${PRINTER_ACCESS_CODE}"
log:
  level: debug
api:
  origin: "*"

Second go2rtc stream config:

streams:
  p1s:
  - ffmpeg:http://biglinux.local:1987/api/stream.mjpeg?src=p1s#video=h264
log:
  level: debug
api:
  origin: "*"

I see a bunch of stuff like this in the logs:

2:34:02.952 PM  debug   [rtsp] handle error=EOF
2:34:02.952 PM  debug   [streams] stop producer url=ffmpeg:http://biglinux.local:1987/api/stream.mjpeg?src=p1s#video=h264
2:34:02.961 PM  debug   [rtsp] disconnect stream=p1s
2:34:12.991 PM  debug   [rtsp] new consumer stream=p1s
2:34:12.991 PM  debug   [exec] run rtsp args=ffmpeg,-hide_banner,-v,error,-fflags,nobuffer,-flags,low_delay,-i,http://biglinux.local:1987/api/stream.mjpeg?src=p1s,-c:v,libx264,-g,50,-profile:v,high,-level:v,4.1,-preset:v,superfast,-tune:v,zerolatency,-pix_fmt:v,yuv420p,-an,-user_agent,ffmpeg/go2rtc,-rtsp_transport,tcp,-f,rtsp,rtsp://127.0.0.1:8554/f30c00d728b14f2a964dcccf6d23c7b8
2:34:14.642 PM  debug   [exec] run rtsp launch=1.651180614s
2:34:14.642 PM  debug   [streams] start producer url=ffmpeg:http://biglinux.local:1987/api/stream.mjpeg?src=p1s#video=h264
2:34:32.986 PM  debug   [rtsp] handle error=EOF
2:34:32.987 PM  debug   [streams] stop producer url=ffmpeg:http://biglinux.local:1987/api/stream.mjpeg?src=p1s#video=h264
2:34:32.994 PM  debug   [rtsp] disconnect stream=p1s
2:34:43.003 PM  debug   [rtsp] new consumer stream=p1s
2:34:43.003 PM  debug   [exec] run rtsp args=ffmpeg,-hide_banner,-v,error,-fflags,nobuffer,-flags,low_delay,-i,http://biglinux.local:1987/api/stream.mjpeg?src=p1s,-c:v,libx264,-g,50,-profile:v,high,-level:v,4.1,-preset:v,superfast,-tune:v,zerolatency,-pix_fmt:v,yuv420p,-an,-user_agent,ffmpeg/go2rtc,-rtsp_transport,tcp,-f,rtsp,rtsp://127.0.0.1:8554/f30c00d728b14f2a964dcccf6d23c7b8
2:34:45.450 PM  debug   [exec] run rtsp launch=2.446903179s
2:34:45.450 PM  debug   [streams] start producer url=ffmpeg:http://biglinux.local:1987/api/stream.mjpeg?src=p1s#video=h264
2:35:03.007 PM  debug   [rtsp] handle error=EOF
2:35:03.007 PM  debug   [streams] stop producer url=ffmpeg:http://biglinux.local:1987/api/stream.mjpeg?src=p1s#video=h264
2:35:03.016 PM  debug   [rtsp] disconnect stream=p1s
2:35:13.025 PM  debug   [rtsp] new consumer stream=p1s
2:35:13.025 PM  debug   [exec] run rtsp args=ffmpeg,-hide_banner,-v,error,-fflags,nobuffer,-flags,low_delay,-i,http://biglinux.local:1987/api/stream.mjpeg?src=p1s,-c:v,libx264,-g,50,-profile:v,high,-level:v,4.1,-preset:v,superfast,-tune:v,zerolatency,-pix_fmt:v,yuv420p,-an,-user_agent,ffmpeg/go2rtc,-rtsp_transport,tcp,-f,rtsp,rtsp://127.0.0.1:8554/f30c00d728b14f2a964dcccf6d23c7b8
2:35:14.818 PM  debug   [exec] run rtsp launch=1.79313451s
2:35:14.818 PM  debug   [streams] start producer url=ffmpeg:http://biglinux.local:1987/api/stream.mjpeg?src=p1s#video=h264
2:35:33.026 PM  debug   [rtsp] handle error=EOF
2:35:33.026 PM  debug   [streams] stop producer url=ffmpeg:http://biglinux.local:1987/api/stream.mjpeg?src=p1s#video=h264
2:35:33.033 PM  debug   [rtsp] disconnect stream=p1s
2:35:43.045 PM  debug   [rtsp] new consumer stream=p1s
2:35:43.045 PM  debug   [exec] run rtsp args=ffmpeg,-hide_banner,-v,error,-fflags,nobuffer,-flags,low_delay,-i,http://biglinux.local:1987/api/stream.mjpeg?src=p1s,-c:v,libx264,-g,50,-profile:v,high,-level:v,4.1,-preset:v,superfast,-tune:v,zerolatency,-pix_fmt:v,yuv420p,-an,-user_agent,ffmpeg/go2rtc,-rtsp_transport,tcp,-f,rtsp,rtsp://127.0.0.1:8554/f30c00d728b14f2a964dcccf6d23c7b8
2:35:45.605 PM  debug   [exec] run rtsp launch=2.560346539s
2:35:45.605 PM  debug   [streams] start producer url=ffmpeg:http://biglinux.local:1987/api/stream.mjpeg?src=p1s#video=h264
WayneManion commented 4 days ago

Here is a video of the "middle" condition where the RTC provides a frame, the screen dims to gray, then another from is presented.

https://www.youtube.com/watch?v=UnGMnuNl4D8