"codecs not match: JPEG" when trying to convert camera snapshot to stream

seanblanchfield commented 1 year ago

First, many thanks for creating this.

I am trying to use a HTTP source to create a stream from a camera snapshot. The snapshot works fine in a web browser, and returns a response with Content-Type image/jpeg. Unfortunately, go2rtc logs a warning as follows:

WRN [rtsp] error="codecs not match: JPEG" stream=kitchen

and does not seem to create the stream.

The camera is actually wall mounted tablet running Fully Kiosk Browser, which provides a HTTP API endpoint for grabbing a camera snapshot (http://10.0.0.21:2323?cmd=getCamshot&password=[PASSWORD]). I have go2rtc running as part of Frigate 12 on a RB4. Everything is working okay with other streams from various IP cameras.

The relevant part of my go2rtc config looks like this:

go2rtc:
  streams:
    kitchen: http://10.0.0.21:2323?cmd=getCamshot&password=[PASSWORD]

Am I doing this incorrectly?

NickM-27 commented 1 year ago

I don't think go2rtc supports http snapshots as a stream, only mjpeg streams. What are you trying to accomplish with a single snapshot, I am not sure that makes sense.

seanblanchfield commented 1 year ago

I was hoping to create some kind of stream, even if it was only 1 FPS. I'm not sure exactly what to expect, but the section in the readme about HTTP sources says:

Support Content-Type: HTTP-JPEG (image/jpeg) - camera snapshot link, can be converted by go2rtc to MJPEG stream

It's a bit brief, but I interpreted that as saying that go2rtc can convert a HTTP URL that returns image/jpeg content into a MJPEG stream.

NickM-27 commented 1 year ago

you are correct, it is working that way. What were you trying to do when this warning happened? as it says it only supports an mjpeg stream from this source

seanblanchfield commented 1 year ago

The warning occurs when I try to open the stream over RTSP from VLC (e.g., using a stream URL like rtsp://10.0.0.2:8554/kitchen where 10.0.0.2 is the address of the host where go2rtc is running). Perhaps VLC cannot play MJPEG, and the warning is effectively saying there are no matching codecs? If that is the case, should I be able to transcode the stream instead?

As an experiment, I just tried transcoding by adjusting my config as follows (based on the MJPEG transcoding example in the readme), but I still get the same warning in the go2rtc logs and VLC fails to open a connection:

  kitchen: 
      - http://10.0.0.21:2323?cmd=getCamshot&password=[PASSWORD]
      - ffmpeg:kitchen#video=h264

AlexxIT commented 1 year ago

Go2rtc doesn't support HTTP-JPEG to RTSP MJPEG yet. Only to HTTP (web). Also it support RTSP MJPEG to RTSP MJPEG.

For transcoding you should use http link and exec source. I saw examples in the internet, how convert static image to stream via ffmpeg. But go2rtc ffmpeg source doesn't support it.

NickM-27 commented 1 year ago

@seanblanchfield should probably setup tinycam

seanblanchfield commented 1 year ago

Tinycam is a good idea, thanks. However, I am mostly exploring the capabilities of go2rtc so I can get the most out of it, especially by centralising all my camera streaming through it.

I figured out how to make it work, helped by Alex's comment that only HTTP-JPEG to HTTP-MJPEG is supported right now. The following configuration creates the HTTP-MJPEG stream and also transcodes the stream to RTSP.

kitchen: 
      - http://10.0.0.21:2323?cmd=getCamshot&password=[PASSWORD]
      - "ffmpeg:http://127.0.0.1:1984/api/stream.mjpeg?src=kitchen#video=h264"

I can then load it as an RTSP stream in VLC or HA (although there's understandably a bit of lag).

I think my confusion up to this point could have been avoided with some extra documentation.

I wasn't using the Web UI because my go2rtc is running as part of Frigate 12. The beta docs for Frigate 12 do not suggest publishing port 1984 in docker. The "fast start" section of the go2rtc readme says to visit the web interface, but that didn't work because the port wasn't published by Frigate. Once I figured this out, the web interface allowed me to explore the streams and better understand how go2rtc works, including that HTTP streams are available over port 1984. Maybe add a "Frigate" option under the "Fast start" section in the readme, and add relevant advice like exposing port 1984.
More forcefully explain that users of go2rtc should refer continuously to the web UI when configuring and debugging streams.
Ask a frigate maintainer to document publishing port 1984 in their example docker-compose.yaml code and explain that anyone using go2rtc should use it to debug streams.
Explicitly state in the section on HTTP streams that these streams will be exposed over HTTP on port 1984.
I was pretty confused about the where the lines are drawn between protocols and codecs. For example, I wasn't thinking of there being a distinction between MJPEG over HTTP vs MJPEG over RTSP. The first part of the readme mentions "streaming to RTSP, WebRTC, MSE/MP4, HLS or MJPEG" but doesn't mention streaming to HTTP. Maybe it would be useful to have a new section that discusses RTSP vs HTTP vs WebRTC vs MSE etc, what ports go2rtc serves these on, and highlights current limitations like HTTP sources only being restreamable via HTTP (not RTSP). This would also be an opportunity to provide sample code for how one might transcode from a HTTP stream from the HTTP API to RTSP.

I'm not yet qualified to write some of this content, but I might be able to help with documentation if that would be useful. I think this is a very important project.

NickM-27 commented 1 year ago

I am one of the frigate maintainers, we've heard similar marks about port 1984 so I will create a PR to add that port mapping to the docs

AlexxIT commented 1 year ago

You confused because don't have access to go2rtc WebUI. There you can find special web page with all useful links to your stream. Including link to MJPEG stream.

As I know, you can access go2rtc WebUI via /live/webrtc url relative to your Frigate WebUI.

markfrancisonly commented 1 year ago

kitchen: 
      - http://10.0.0.21:2323?cmd=getCamshot&password=[PASSWORD]
      - "ffmpeg:http://127.0.0.1:1984/api/stream.mjpeg?src=kitchen#video=h264"

I'm also wanting to pull fully browser tablet camera snapshots into frigate, and this workaround is basically not working in my setup because my recorded clips are saved at 1/4 speed for some reason, and my i7 cpu usage has increased 10% for each http jpeg source vs less than 2% for each rtsp cameras source.

also I don't seem to be able to override the default ffmpeg http loop framerate (25) in go2rtc. I'd prefer to set the polling loop to 1-4 fps. The go2rtc rtsp stream is 25fps which doesn't make sense for this particular image source.

edit: forgot to mention I'm running go2rtc 1.5 which is capable of streaming the http source as mjpeg but then the frigate ui won't play back the recorded clips, which then requires vlc to playback recordings

NickM-27 commented 1 year ago

@markfrancisonly one thing is you most likely would want #hardware there otherwise the CPU is doing software encoding which would explain increased cpu usage

edit: forgot to mention I'm running go2rtc 1.5 which is capable of streaming the http source as mjpeg but then the frigate ui won't play back the recorded clips, which then requires vlc to playback recordings

you need to add ?video=h264 to the url in the frigate config that is requesting the stream

markfrancisonly commented 1 year ago

check out this recorded segment, tried the #hardware and ?video=h264 params without effect

https://github.com/AlexxIT/go2rtc/assets/12145270/55e42850-131d-4327-8d8f-8c9fd7e69554

NickM-27 commented 1 year ago

Very weird, I helped someone else with a similar setup and it worked well; though I believe mjpeg has a number of different factors that can cause different outcomes

markfrancisonly commented 1 year ago

my mistake, #hardware did drop the cpu usage. recorded video is still 1/4 speed. the live jsmpeg view is full speed?

@NickM-27 one more thing, the segments are saved every 3 seconds instead of the usual 10 seconds. e.g. 22.00.mp4 22.03.mp4 22.06.mp4 22.09.mp4 vs. 22.00.mp4 22.10.mp4 22.20.mp4 22.30.mp4

NickM-27 commented 1 year ago

jsmpeg is the detect fps and capped at 10 fps.

Are the recordings h264 or mipeg?

AlexxIT commented 1 year ago

Starting from v1.2.0 go2rtc supports HTTP MJPEG to RTSP MJPEG: https://github.com/AlexxIT/go2rtc/releases/tag/v1.2.0

seanblanchfield commented 1 year ago

Thanks for your work on this @AlexxIT . I tried again, but I can't get it to work. Maybe you can let me know if I'm getting it wrong. If I understand correctly, the stream as I had it configured in the first post above should now work, i.e.,:

  streams:
    kitchen: http://10.0.0.21:2323?cmd=getCamshot&password=[PASSWORD]

On the links page in go2rtc I see the stream, and all the MJPEG sources work perfectly. However, the H264/H265 and RTSP links do not work. In VLC, I get the following messages logged when I try to view the RTSP stream:

live555 error: Failed to connect with rtsp://10.0.0.2:8554/kitchen
satip error: Failed to setup RTSP session

(where 10.0.0.2 is the host I'm running go2rtc on)

And the corresponding log message from go2rtc is:

WRN [rtsp] error="unknown error" stream=kitchen

Other RTSP streams that are being proxied through the same instance of go2rtc are working fine, and open successfully in VLC.

Am I missing something, or is there anything I can do to help analyse this?

AlexxIT commented 1 year ago

You need to increase log level to trace and check logs

seanblanchfield commented 1 year ago

Thanks Alex. I increased the log level in the go2rtc config to "debug" and tried again. The go2rtc log output when I try to connect to the RTSP stream is now:

2023-08-23 09:07:37.159186311  09:07:37.158 DBG [rtsp] new consumer stream=kitchen
2023-08-23 09:07:37.418225675  09:07:37.418 DBG [streams] stop producer url=http://10.0.0.21:2323/?cmd=getCamshot&password=[PASSWORD]
2023-08-23 09:07:37.418657809  09:07:37.418 WRN [rtsp] error="unknown error" stream=kitchen

The logged output in VLC is still the same.

Does the above give you any info to work on? Let me know if there's anything else I can do.

AlexxIT commented 1 year ago

Try to "trace" log level

seanblanchfield commented 1 year ago

Thanks Alex. I enabled "trace" logging and repeated the attempt to stream the RTSP stream via VLC. Here's what I got (using v1.6.2):

2.531591295  15:20:52.531 TRC [rtsp] server request:
2023-08-23 15:20:52.531603062  OPTIONS rtsp://10.0.0.2:8554/kitchen RTSP/1.0
2023-08-23 15:20:52.531607910  Cseq: 2
2023-08-23 15:20:52.531613664  User-Agent: LibVLC/3.0.18 (LIVE555 Streaming Media v2016.11.28)
2023-08-23 15:20:52.531618006  
2023-08-23 15:20:52.531622314  
2023-08-23 15:20:52.531627192  15:20:52.531 TRC [rtsp] server response:
2023-08-23 15:20:52.531631469  RTSP/1.0 200 OK
2023-08-23 15:20:52.531636444  Public: OPTIONS, SETUP, TEARDOWN, DESCRIBE, PLAY, PAUSE, ANNOUNCE, RECORD
2023-08-23 15:20:52.531640553  Cseq: 2
2023-08-23 15:20:52.531644710  
2023-08-23 15:20:52.531648644  
2023-08-23 15:20:52.533290319  15:20:52.533 TRC [rtsp] server request:
2023-08-23 15:20:52.533300271  DESCRIBE rtsp://10.0.0.2:8554/kitchen RTSP/1.0
2023-08-23 15:20:52.533305132  Cseq: 3
2023-08-23 15:20:52.533311086  User-Agent: LibVLC/3.0.18 (LIVE555 Streaming Media v2016.11.28)
2023-08-23 15:20:52.533315661  Accept: application/sdp
2023-08-23 15:20:52.533319577  
2023-08-23 15:20:52.533323557  
2023-08-23 15:20:52.533328553  15:20:52.533 DBG [rtsp] new consumer stream=kitchen
2023-08-23 15:20:52.533380623  15:20:52.533 TRC [streams] check cons=0 media=video, sendonly, H264, H265
2023-08-23 15:20:52.752557410  15:20:52.752 TRC [streams] check prod=0 media=video, recvonly, JPEG
2023-08-23 15:20:52.753223085  15:20:52.752 TRC [streams] check cons=0 media=audio, sendonly, MPEG4-GENERIC
2023-08-23 15:20:52.753254981  15:20:52.752 TRC [streams] check prod=0 media=video, recvonly, JPEG
2023-08-23 15:20:52.753436593  15:20:52.753 DBG [streams] stop producer url=http://10.0.0.21:2323/?cmd=getCamshot&password=[PASSWORD]
2023-08-23 15:20:52.753729055  15:20:52.753 WRN [rtsp] error="unknown error" stream=kitchen
2023-08-23 15:20:52.753842568  15:20:52.753 TRC [rtsp] server response:
2023-08-23 15:20:52.753850464  RTSP/1.0 404 Not Found
2023-08-23 15:20:52.753856099  Cseq: 3
2023-08-23 15:20:52.753861639  
2023-08-23 15:20:52.753867931  
2023-08-23 15:20:52.784435269  15:20:52.784 TRC [rtsp] server request:
2023-08-23 15:20:52.784449464  SETUP rtsp://10.0.0.2:8554/kitchen RTSP/1.0
2023-08-23 15:20:52.784456944  Cseq: 0
2023-08-23 15:20:52.784464588  Transport: RTP/AVP;unicast;client_port=9046-9047
2023-08-23 15:20:52.784470953  
2023-08-23 15:20:52.784477602  
2023-08-23 15:20:52.784547561  15:20:52.784 TRC [rtsp] server response:
2023-08-23 15:20:52.784556661  RTSP/1.0 461 Unsupported transport
2023-08-23 15:20:52.784563911  Cseq: 0
2023-08-23 15:20:52.784570530  
2023-08-23 15:20:52.784576793  
2023-08-23 15:20:52.791354681  15:20:52.790 TRC [rtsp] server request:
2023-08-23 15:20:52.791369127  OPTIONS rtsp://10.0.0.2:8554 RTSP/1.0
2023-08-23 15:20:52.791379258  Clientid: Linux_2.4_6.0.9.1235_play32_RN01_EN_586
2023-08-23 15:20:52.791386985  Cseq: 1
2023-08-23 15:20:52.791396674  User-Agent: RealMedia Player Version 6.0.9.1235 (linux-2.0-libc6-i386-gcc2.95)
2023-08-23 15:20:52.791405476  Clientchallenge: 9e26d33f2984236010ef6253fb1887f7
2023-08-23 15:20:52.791414197  Playerstarttime: [28/03/2003:22:50:23 00:00]
2023-08-23 15:20:52.791420721  Companyid: KnKV4M4I/B2FjJ1TToLycw==
2023-08-23 15:20:52.791427288  Guid: 00000000-0000-0000-0000-000000000000
2023-08-23 15:20:52.791433401  Regiondata: 0
2023-08-23 15:20:52.791439299  
2023-08-23 15:20:52.791444913  
2023-08-23 15:20:52.792135414  15:20:52.791 TRC [rtsp] server response:
2023-08-23 15:20:52.792145564  RTSP/1.0 200 OK
2023-08-23 15:20:52.792155034  Public: OPTIONS, SETUP, TEARDOWN, DESCRIBE, PLAY, PAUSE, ANNOUNCE, RECORD
2023-08-23 15:20:52.792162114  Cseq: 1
2023-08-23 15:20:52.792168814  
2023-08-23 15:20:52.792175091

AlexxIT commented 1 year ago

Unfortunately nothing useful between

2023-08-23 15:20:52.753436593  15:20:52.753 DBG [streams] stop producer url=http://10.0.0.21:2323/?cmd=getCamshot&password=[PASSWORD]
2023-08-23 15:20:52.753729055  15:20:52.753 WRN [rtsp] error="unknown error" stream=kitchen

Can you provide access to link in PM? https://github.com/AlexxIT/go2rtc/wiki/Tunnel-RTSP-camera-to-Intenet You can check my contacts in the github profile

seanblanchfield commented 1 year ago

Thanks. I've DM'ed you tunnel details on telegram.

seanblanchfield commented 1 year ago

Alex was extremely helpful via Telegram earlier today. We discovered that he was able to view the problem stream using VLC 3.0.18 on his Mac, while I was unable to using VLC 3.0.18 on my Ubuntu laptop. On that basis it seemed like the problem was with VLC.

I decided to do some more testing, so I installed VLC 3.0.18 on my wife's Macbook and tried again, but it didn't work. I also tried on VLC for Android, but still no success.

Alex mentioned that he was testing using ffmpeg/ffplay, so I installed that. Other RTSP streams from go2rtc would work, but the "kitchen" stream did not. However, ffplay gave a more useful error:

rtsp @ 0x7f2474000cc0] method DESCRIBE failed: 404 Not Found=0/0   
rtsp://10.0.0.2:8554/kitchen: Server returned 404 Not Found

This prompted me to try to capture the traffic between ffplay and go2rtc with wireshark. Here is the "conversation" starting with the client requesting "OPTIONS", getting an answer, then requesting "DESCRIBE" and getting a 404.

OPTIONS rtsp://10.0.0.2:8554/kitchen RTSP/1.0
CSeq: 1
User-Agent: Lavf58.76.100

RTSP/1.0 200 OK
Cseq: 1
Public: OPTIONS, SETUP, TEARDOWN, DESCRIBE, PLAY, PAUSE, ANNOUNCE, RECORD

DESCRIBE rtsp://10.0.0.2:8554/kitchen RTSP/1.0
Accept: application/sdp
CSeq: 2
User-Agent: Lavf58.76.100

RTSP/1.0 404 Not Found
Cseq: 2

I repeated this capture while trying with VLC and I get exactly the same traffic capture: a 404 to the second request.

This isn't mission critical or urgent for me, and Alex pointed out that I would be better off using the HTTP-MJPEG stream (which works) because it is less CPU-intensive. However, it looks like a bug, so I thought I'd contribute as much info as I could.

If there's a different build of go2rtc you would like me to try, just let me know.

AlexxIT commented 1 year ago

404 because stream fails to start. But we don't know why it fails.

Loic691 commented 1 month ago

Hi all Do you provide solution for this issue ? I'd like to convert HTTP JPEG to H264 with go2rtc for webrtc but also MJPEG view is available in GO2RTC web gui...

AlexxIT commented 1 month ago

https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#source-ffmpeg

Loic691 commented 1 month ago

May I have to understand no ? I have the same issue that @seanblanchfield When I request in go2rtc this config, I have only the MJPEG stream and no WEBRTC neither no RTSP

tablette_entree_webrtc:
      - http://192.168.2.170:2323/?cmd=getCamshot&password=XXXXX;
      #- ffmpeg:http://127.0.0.1:1984/api/stream.mjpeg?src=tablette_entree_webrtc#video=h264" => 404 Error
       - ffmpeg:tablette_entree_webrtc#video=h264#hardware

Note that the HTTP MJPEG stream generate by go2rtc is OK in VLC and web browser

AlexxIT commented 1 month ago

Better to take direct MJPEG link from go2rtc and use it as input for ffmpeg source. You have commented this line.

Loic691 commented 1 month ago

Better to take direct MJPEG link from go2rtc and use it as input for ffmpeg source. You have commented this line.

As mentionned by @seanblanchfield , when I try to use as input to ffmpeg the MJPEG link from go2rtc, I have a 404 error...

AlexxIT / go2rtc

"codecs not match: JPEG" when trying to convert camera snapshot to stream #246