Closed fuatakgun closed 1 year ago
You can see how we are using it in frigate: https://github.com/blakeblackshear/frigate/blob/b1ec56de29b7ee0017954b0b4a841ac06def068b/frigate/restream.py#L47
Thanks a lot, let me check.
How ffmpeg can produce RTSP from TCP? It should know codec. It can't know codec from your example
ffmpeg -i tcp:ip:port -c copy -rtsp_transport tcp -f rtsp {output}
You are right, I missed the important stuff and focused on my question.
I will be sending this
ffmpeg -f h264 -i tcp:ip:port -c copy -rtsp_transport tcp -f rtsp {output}
When I call the streams api with PUT, would it update the existing one, in case there is one exists? or I have to call DELETE first and PUT afterwards?
As a follow up question, can we set ffmpeg
as tcp server and can I connect to server and send the data to it (without me setting up the server)?
As of now, I am setting up tcp server, ffmpeg connects to it and client handler sends the data to ffmpeg.
Put will overwrite stream with new object. This is not good idea if you have active connections on current stream. Them continue to work, but you can't see it in streams list.
I think I can fix it in the future, but better to have issue about this problem.
Current overriding is good for me, as i will probably write it once on every restart of HA.
About ffmpeg, tcp server, i am receiving video data as bytes over a websocket connection so there is no path.
I had used ffmpeg with pipe ar first but it was very unreliable.
Then started ffmpeg with tcp, but handling ffmpeg process inside home assistant is very tricky.
If i can outsource handling ffmpeg + tcp server and just write the bytes to server, life would be easier.
Just a side note, these devices are not streaming all the time, they are running on demand to preserve battery life.
Do you have two tracks for video and audio? Do you know what inside video data? This is H264 Annex-B stream? Can you send some dumps?
Now I plan to build in support for receiving MJPEG data via TCP. Maybe I should add support for your data. This support will be built into go2rtc. There will be no need to use FFmpeg.
I was thingking something like this:
command = f"exec: ffmpeg -re -y -f {video_codec} -i tcp://{ip}:{port} -vcodec copy -rtsp_transport tcp -f rtsp {output}"
Interesting idea. Who is init connection?
echo
source, and run exec/ffmpeg with your example (but from echo source)Or maybe:
But go2rtc will not run ffmpeg if no one is going to watch it...
you are right, my proposal seems to be problematic given if user does not start watching the stream in timely manner, frames (bytes) will not be written to ffmpeg, right?
any ideas or suggestions other than initiating the stream automatically?
But why some one needs stream that not viewed by anyone? :)
But anyway. go2rtc support "external" streams. You need to create stream with empty source and later push data to it with RTSP. You need to start ffmpeg manually, not form go2rtc.
streams:
camera1:
# ffmpeg -re -f {video_codec} -i tcp://{ip}:{port} -vcodec copy -rtsp_transport tcp -f rtsp rtsp://localhost:8554/camera1
PS. -f
it is format, not codec. What values you plan to use there?
I really do not want to manage the ffmpeg process from python, it is bringing me to the initial stage (using rtsp simple server)
if i start the stream after setting the api/stream with camera and exec code, would it work?
For codec, i will use h264 or h265
go2rtc won't run exec/ffmpeg just after you add stream via API. Someone needs to request this stream from go2rtc via RTSP or MSE or WebRTC... And go2rtc will stop ffmpeg when client will stop viewing stream.
Maybe I can add new WebSocket API to go2rtc for receiving frames via WebSocket. This does not require ffmpeg at all. But I need some kind of packet dump that your software gets from the cloud WebSocket.
Or you can give me remote access to one of these cameras. As I can understand we're talking about https://github.com/fuatakgun/eufy_security ?
yes, you are right, it is the repository. there is another developer who is extracting the bytes from another docker (eufy-security-ws) and sending the bytes over websocket. I am forwarding incoming bytes to RTSP server and giving back the RTSP URL to home assistant, so users can play live stream from their cameras.
Initial test setup, which is perfectly working fine.
ffmpeg -re -i bunny.mp4 -vcodec copy -f mpegts tcp://192.168.178.119:1234?listen -loglevel debug
ffmpeg -re -y -f mpegts -i tcp://192.168.178.119:1234 -vcodec copy output.mp4
Testing with go2rtc;
streams:
cam: exec:ffmpeg -re -i tcp://192.168.178.119:1234 -vcodec copy -f rtsp -rtsp_transport tcp {output}
ffmpeg -re -i bunny.mp4 -vcodec copy -f mpegts tcp://192.168.178.119:1234?listen -loglevel debug
ffprobe rtsp://192.168.178.119:8554/cam
broken_pipe error
, I was expecting this to work out but it is failing, here are my console output;- console 1:
ffmpeg version 2022-07-18-git-cb22d5ea3c-essentials_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
libavutil 57. 29.100 / 57. 29.100
libavcodec 59. 39.100 / 59. 39.100
libavformat 59. 29.100 / 59. 29.100
libavdevice 59. 8.101 / 59. 8.101
libavfilter 8. 46.100 / 8. 46.100
libswscale 6. 8.100 / 6. 8.100
libswresample 4. 8.100 / 4. 8.100
libpostproc 56. 7.100 / 56. 7.100
Splitting the commandline.
Reading option '-re' ... matched as option 're' (read input at native frame rate; equivalent to -readrate 1) with argument '1'.
Reading option '-i' ... matched as input url with argument 'bunny.mp4'.
Reading option '-vcodec' ... matched as option 'vcodec' (force video codec ('copy' to copy stream)) with argument 'copy'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'mpegts'.
Reading option 'tcp://192.168.178.119:1234?listen' ... matched as output url.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-g' ... matched as AVOption 'g' with argument '15'.
Trailing option(s) found in the command: may be ignored.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url bunny.mp4.
Applying option re (read input at native frame rate; equivalent to -readrate 1) with argument 1.
Successfully parsed a group of options.
Opening an input file: bunny.mp4.
[NULL @ 000001d5b327ebc0] Opening 'bunny.mp4' for reading
[file @ 000001d5b327b800] Setting default whitelist 'file,crypto,data'
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001d5b327ebc0] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001d5b327ebc0] ISO: File Type Major Brand: isom
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001d5b327ebc0] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001d5b327ebc0] Processing st: 0, edit list 0 - media time: 1024, duration: 153600
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001d5b327ebc0] Offset DTS by 1024 to make first pts zero.
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001d5b327ebc0] Setting codecpar->delay to 2 for stream st: 0
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001d5b327ebc0] Before avformat_find_stream_info() pos: 4502 bytes read:32768 seeks:0 nb_streams:1
[h264 @ 000001d5b328f040] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 000001d5b328f040] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 000001d5b328f040] nal_unit_type: 7(SPS), nal_ref_idc: 3
[h264 @ 000001d5b328f040] nal_unit_type: 8(PPS), nal_ref_idc: 3
[h264 @ 000001d5b328f040] nal_unit_type: 6(SEI), nal_ref_idc: 0
[h264 @ 000001d5b328f040] nal_unit_type: 5(IDR), nal_ref_idc: 3
[h264 @ 000001d5b328f040] Format yuv420p chosen by get_format().
[h264 @ 000001d5b328f040] Reinit context to 1280x720, pix_fmt: yuv420p
[h264 @ 000001d5b328f040] no picture
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001d5b327ebc0] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001d5b327ebc0] After avformat_find_stream_info() pos: 591656 bytes read:591656 seeks:0 frames:1
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'bunny.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : Big Buck Bunny, Sunflower version
artist : Blender Foundation 2008, Janus Bager Kristensen 2013
composer : Sacha Goedegebure
encoder : Lavf57.63.100
comment : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
genre : Animation
Duration: 00:00:10.00, start: 0.000000, bitrate: 25657 kb/s
Stream #0:0[0x1](und), 1, 1/15360: Video: h264 (High), 1 reference frame (avc1 / 0x31637661), yuv420p(progressive, left), 1280x720 [SAR 1:1 DAR 16:9], 0/1, 25653 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
Successfully opened the file.
Applying option f (force format) with argument mpegts.
Successfully parsed a group of options.
Opening an output file: tcp://192.168.178.119:1234?listen.
[tcp @ 000001d5b3701f40] No default whitelist set
Successfully opened the file.
[mpegts @ 000001d5b3285180] service 1 using PCR in pid=256, pcr_period=100ms
[mpegts @ 000001d5b3285180] muxrate VBR, sdt every 500 ms, pat/pmt every 100 ms
Output #0, mpegts, to 'tcp://192.168.178.119:1234?listen':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : Big Buck Bunny, Sunflower version
artist : Blender Foundation 2008, Janus Bager Kristensen 2013
composer : Sacha Goedegebure
genre : Animation
comment : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
encoder : Lavf59.29.100
Stream #0:0(und), 0, 1/90000: Video: h264 (High), 1 reference frame (avc1 / 0x31637661), yuv420p(progressive, left), 1280x720 (0x0) [SAR 1:1 DAR 16:9], 0/1, q=2-31, 25653 kb/s, 30 fps, 30 tbr, 90k tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
Automatically inserted bitstream filter 'h264_mp4toannexb'; args=''
av_interleaved_write_frame(): Unknown errortime=00:00:02.06 bitrate=25839.8kbits/s speed=1.01x
No more output streams to write to, finishing.
Error writing trailer of tcp://192.168.178.119:1234?listen: Error number -10053 occurred
frame= 80 fps= 31 q=-1.0 Lsize= 8468kB time=00:00:02.56 bitrate=27027.5kbits/s speed=1.01x
video:8276kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.322037%
Input file #0 (bunny.mp4):
Input stream #0:0 (video): 80 packets read (8474585 bytes);
Total: 80 packets (8474585 bytes) demuxed
Output file #0 (tcp://192.168.178.119:1234?listen):
Output stream #0:0 (video): 80 packets muxed (8474585 bytes);
Total: 80 packets (8474585 bytes) muxed
[AVIOContext @ 000001d5b3ab48c0] Statistics: 8671368 bytes written, 0 seeks, 306 writeouts
Error closing file tcp://192.168.178.119:1234?listen: Error number -10053 occurred
0 frames successfully decoded, 0 decoding errors
[AVIOContext @ 000001d5b327c300] Statistics: 8479087 bytes read, 0 seeks
Conversion failed!
go2rtc logs;
config-go2rtc-1 | 16:56:34.526 DBG [exec] run url="exec:ffmpeg -re -i tcp://192.168.178.119:1234 -vcodec copy -f rtsp -rtsp_transport tcp rtsp://localhost:8554/eefac2a7a8ec8616ac8a48be5fe42409"
config-go2rtc-1 | ffmpeg version 5.0.1 Copyright (c) 2000-2022 the FFmpeg developers
config-go2rtc-1 | built with gcc 11.2.1 (Alpine 11.2.1_git20220219) 20220219
config-go2rtc-1 | configuration: --prefix=/usr --enable-avfilter --enable-gnutls --enable-gpl --enable-libass --enable-libmp3lame --enable-libpulse --enable-libvorbis --enable-libvpx --enable-libxvid --enable-libx264 --enable-libx265 --enable-libtheora --enable-libv4l2 --enable-libdav1d --enable-lto --enable-postproc --enable-pic --enable-pthreads --enable-shared --enable-libxcb --enable-librist --enable-libsrt --enable-libssh --enable-libvidstab --disable-stripping --disable-static --disable-librtmp --enable-libaom --enable-libopus --enable-libsoxr --enable-libwebp --enable-vaapi --enable-vdpau --enable-vulkan --enable-libdrm --disable-debug
config-go2rtc-1 | libavutil 57. 17.100 / 57. 17.100
config-go2rtc-1 | libavcodec 59. 18.100 / 59. 18.100
config-go2rtc-1 | libavformat 59. 16.100 / 59. 16.100
config-go2rtc-1 | libavdevice 59. 4.100 / 59. 4.100
config-go2rtc-1 | libavfilter 8. 24.100 / 8. 24.100
config-go2rtc-1 | libswscale 6. 4.100 / 6. 4.100
config-go2rtc-1 | libswresample 4. 3.100 / 4. 3.100
config-go2rtc-1 | libpostproc 56. 3.100 / 56. 3.100
config-go2rtc-1 | Input #0, mpegts, from 'tcp://192.168.178.119:1234':
config-go2rtc-1 | Duration: N/A, start: 1.466667, bitrate: N/A
config-go2rtc-1 | Program 1
config-go2rtc-1 | Metadata:
config-go2rtc-1 | service_name : Big Buck Bunny, Sunflower version
config-go2rtc-1 | service_provider: FFmpeg
config-go2rtc-1 | Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn
config-go2rtc-1 | 16:56:36.212 DBG [exec] run launch=1.6855205s
config-go2rtc-1 | Output #0, rtsp, to 'rtsp://localhost:8554/eefac2a7a8ec8616ac8a48be5fe42409':
config-go2rtc-1 | Metadata:
config-go2rtc-1 | encoder : Lavf59.16.100
config-go2rtc-1 | Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 30 tbr, 90k tbn
config-go2rtc-1 | Stream mapping:
config-go2rtc-1 | Stream #0:0 -> #0:0 (copy)
config-go2rtc-1 | Press [q] to stop, [?] for help
av_interleaved_write_frame(): Broken pipe00:00:00.53 bitrate=N/A speed=1.06x =N/A speed=N/A
config-go2rtc-1 | frame= 30 fps=0.0 q=-1.0 Lsize=N/A time=00:00:00.90 bitrate=N/A speed=1.03x
config-go2rtc-1 | video:2762kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
config-go2rtc-1 | Conversion failed!
console 2;
ffprobe version 2022-07-18-git-cb22d5ea3c-essentials_build-www.gyan.dev Copyright (c) 2007-2022 the FFmpeg developers
built with gcc 12.1.0 (Rev2, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
libavutil 57. 29.100 / 57. 29.100
libavcodec 59. 39.100 / 59. 39.100
libavformat 59. 29.100 / 59. 29.100
libavdevice 59. 8.101 / 59. 8.101
libavfilter 8. 46.100 / 8. 46.100
libswscale 6. 8.100 / 6. 8.100
libswresample 4. 8.100 / 4. 8.100
libpostproc 56. 7.100 / 56. 7.100
[rtsp @ 0000025c2b200e40] method SETUP failed: 461 Unsupported transport
[h264 @ 0000025c2b204800] co located POCs unavailable
Input #0, rtsp, from 'rtsp://192.168.178.119:8554/cam':
Duration: N/A, start: 0.066667, bitrate: N/A
Stream #0:0: Video: h264 (High), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn
Your example works for me. But only once. When user stop watch stream - go2rtc will close ffmpeg. And second ffmpeg also will fail.
# ffmpeg -re -i bbb.mp4 -c:v copy -f mpegts tcp://localhost:1234?listen
streams:
cam1: exec:ffmpeg -re -i tcp://localhost:1234 -c:v copy -rtsp_transport tcp -f rtsp {output}
thanks again for your great work with webrtc custom card and this add-on.
Apparently, there is an endpoint working for Home Assistant integration to add stream (with names and URLs) on the fly.
What is the API endpoint for the same process? I would like to add cameras on the fly like this;
name: camera_name source: ffmpeg -i tcp:ip:port -c copy -rtsp_transport tcp -f rtsp {output}
I am receiving video bytes inside home assistant through eufy security custom integration and I want to send these bytes to respective tcp port and let go2rtc handle ffmpeg process and generate the rtsp.
currently, I am setting up tcp server, creating a ffmpeg process to connect to tcp server and publish to
rtsp-simple-server
for rtsp streaming. I want to get rid of ffmpeg process management.