gtxaspec commented 1 year ago

initial testing seems ok, 1080p/360p 30fps OK via webrtc / hls web streaming, go2rtc binary running on device.

rtsp OK

audio works, aac+rtsp, libopus+webrtc

test with frigate: frigate recordings are good.

if testing is good, we will deprecate v4l2rtspserver in favor of go2rtc.

gtxaspec commented 1 year ago

added in latest master. feel free to leave feedback here.

gtxaspec commented 1 year ago

example to view stream in browser: http://192.168.x.x:1984/stream.html?src=1080p&mode=webrtc still: http://192.168.x.x:1984/api/frame.mp4?src=1080p stream links: http://192.168.x.x:1984/links.html?src=1080p

endertable commented 1 year ago

So the v4l2rtspserver can be completely disabled? Will the cmd commands run without it running?

gtxaspec commented 1 year ago

Yes, and yes.

They are independent of each other.

rickgitdone commented 1 year ago

is there a new build we can download to switch over to webrtc? or it is already ready and we just pull new git?

gtxaspec commented 1 year ago

Download master, it's ready

KyleStilkey commented 1 year ago

I've updated my cameras and I can't seem to figure out how to use this new release, is there a setting that must be enabled?

I've tried your example links and nothing comes up.

http://(Cams IP):1984/stream.html?src=1080p&mode=webrtc and nothing shows up.

gtxaspec commented 1 year ago

What model camera? Does the web UI show up at http://(Cams IP):1984/ ?

Can you share your wz_mini.conf?

KyleStilkey commented 1 year ago

I have V3's I'm using and no, I do not get a web UI at all going to that URL for the cameras.

This is the wz_mini.cof from the cameras.

wz_mini configuration

W E B CAMERA

THIS MODE DISABLES EVERYTHING AND IT WILL

WORK AS A WEB CAMERA FOR YOUR PC ONLY

WEB_CAM_ENABLE="false" WEB_CAM_BIT_RATE="8000" WEB_CAM_FPS_RATE="25"

NETWORKING

CUSTOM_HOSTNAME="WCV3"

ENABLE_USB_ETH="false" ENABLE_USB_ETH_MODULE_AUTODETECT="true" ENABLE_USB_ETH_MODULE_MANUAL=""

ENABLE_USB_DIRECT="false" USB_DIRECT_MAC_ADDR="02:01:02:03:04:08"

ENABLE_USB_RNDIS="false"

ENABLE_IPV6="false"

ENABLE_WIREGUARD="false" WIREGUARD_IPV4="" WIREGUARD_PEER_ENDPOINT="" WIREGUARD_PEER_PUBLIC_KEY="" WIREGUARD_PEER_ALLOWED_IPS="" WIREGUARD_PEER_KEEP_ALIVE=""

ENABLE_IPTABLES="false"

ENABLE_NFSv4="false"

ENABLE_RTL8189FS_DRIVER="true" ENABLE_ATBM603X_DRIVER="true"

NETWORK INTERFACE BONDING

BONDING_ENABLED="false" BONDING_PRIMARY_INTERFACE="eth0" BONDING_SECONDARY_INTERFACE="wlan0" BONDING_LINK_MONITORING_FREQ_MS="100" BONDING_DOWN_DELAY_MS="5000" BONDING_UP_DELAY_MS="5000" BONDING_FAIL_OVER_MAC="0"

ACCESSORIES

REMOTE_SPOTLIGHT="false" REMOTE_SPOTLIGHT_HOST="0.0.0.0"

VIDEO STREAM

RTSP_LOGIN="admin" RTSP_PASSWORD="wyze" RTSP_PORT="8554"

RTSP_HI_RES_ENABLED="true" RTSP_HI_RES_ENABLE_AUDIO="true" RTSP_HI_RES_FPS="" RTSP_HI_RES_MAX_BITRATE="" RTSP_HI_RES_TARGET_BITRATE="" RTSP_HI_RES_ENC_PARAMETER=""

RTSP_LOW_RES_ENABLED="false" RTSP_LOW_RES_ENABLE_AUDIO="false" RTSP_LOW_RES_FPS="" RTSP_LOW_RES_MAX_BITRATE="" RTSP_LOW_RES_TARGET_BITRATE="" RTSP_LOW_RES_ENC_PARAMETER=""

RTSP_AUTH_DISABLE="true"

RTMP STREAMING

REQUIRES RTSP

AND RTSP AUDIO

RTMP_STREAM_ENABLED="false" RTMP_STREAM_FEED="video1_unicast" RTMP_STREAM_SERVICE="youtube" RTMP_STREAM_DISABLE_AUDIO="false" RTMP_STREAM_YOUTUBE_KEY="" RTMP_STREAM_TWITCH_KEY="" RTMP_STREAM_FACEBOOK_KEY=""

GENERAL

LIBCALLBACK_ENABLE="true" ENABLE_SWAP="true" ENABLE_USB_STORAGE="false" ENABLE_EXT4="false" ENABLE_CIFS="false" DISABLE_FW_UPGRADE="true" AUDIO_PROMPT_VOLUME="50" ENABLE_MP4_WRITE="false" NIGHT_DROP_DISABLE="false" DISABLE_MOTOR="false" ENABLE_FSCK_ON_BOOT="false" ENABLE_CAR_DRIVER="false" ENABLE_LOCAL_DNS="false" ENABLE_CRONTAB="false"

DEBUG

drops you to a shell via serial, doesn't load app_init.sh

DEBUG_ENABLED="false" DEBUG_INITRAMFS_ENABLED="false" DEBUG_PASSWORD="false" ENABLE_SYSLOG_SAVE="false"

WEB

WEB_SERVER_ENABLED="false"

SCRIPTING

CUSTOM_SCRIPT_PATH=""

gtxaspec commented 1 year ago

you have an old version of wz_mini.conf.

you can either do a fresh install, or copy /opt/wz_mini/etc/wz_mini.conf.d to /opt/wz_mini/wz_mini.conf (this will overwrite your old config) to get the new go2rtc options.

mega3st commented 1 year ago

@gtxaspec first of all thanks for implementing go2rtc as a potential replacement for v4l2rtspserver!

long story short after lots of testing with tinycam ( #340):

it seems like tinycam doesn’t like the aac_he profile
but it is flawlessly working with low latency and no stuttering by using opus with the following parameters

- exec:ffmpeg -hide_banner -f alsa -ac 1 -i dsnooper0 -c:a libopus -compression_level 4 -frame_duration 10 -muxdelay 0 -muxpreload 0 -ac:a 1 -af adelay=0|0 -f rtsp {output}

for compression_level values between 0 and 10 are allowed
- 0 gives fastet encoding but lowest audio quality
- 10 is the default value and gives slowest encoding but best audio quality (this should also be the reason for stuttering with the current config)
- not sure if 4 is already the sweetspot but for now it is more than good enough
for -frame duration the values 2.5, 5, 10, 20, 40, 60 are allowed
- 20 ms is the default value with low packet overhead and high audio quality
- 10 ms is according to the docs the sweetspot for low latency with the best compromise between audio quality and packet overhaed
- initial test with compression_level 5 and 20 ms frame duration resulted in high latency
- second test with compression_level 4 and 10 ms frame duration works like a charm

@mrlt8 maybe this is also interesting for your wyze-bridge

Other findings while testing

if more than one audio output is defined in go2rtc.yml and "audio=all" is set in the rtsp link tinycam picks the first defined audio output _(currenty pcms16le)
it therefore makes sense to use specific instead of generic rtsp link e.g. rtsp://[IP-ADDRESS]:8554/1080p?video=h264&audio=opus instead of rtsp://[IP-ADDRESS]:8554/1080p?video=all&audio=all
no matter what parameters I’ve tried the result for aac_he with tinycam was at best an established audio connection with only short random noises
the default aac profile (lc) with the following configuration worked immediately but had very high latency

- exec:ffmpeg -hide_banner -f alsa -ac 1 -i dsnooper0 -c:a libfdk_aac -afterburner 1 -channels 1 -vbr 3 -muxdelay 0 -muxpreload 0 -ac:a 1 -f rtsp {output}

CONCLUSION

At least from my perspective go2rtc is working fantastic and after changing back to opus as default audio output or adding some notes for tinycam to the wiki I see no reason not to deprecate v4l2rtspserver!

gtxaspec commented 1 year ago

Have you tried with the regular aac codec? (Not libfdk)

mega3st commented 1 year ago

No, but since I’m going to figure out the lowest compression level for opus that still delivers audio quality on par with the original stream during the weekend, it shouldn’t be a problem to do some further testing wit aac as well

gtxaspec commented 1 year ago

I've got some other latency fixes coming, if they work out.

Hopefully we can eliminate using v4l2loopback.ko and ALSA altogether, and pipe them straight to ffmpeg.

mega3st commented 1 year ago

Latency fixes are always welcome and cutting out v4l2loopback.ko and ALSA sounds like big potential - really looking forward to test them!

Below my additional test results.

All tests were performed with wyzeCam V3, tinyCam app on Android 13 an the following settings:

GO2RTC_SERVER_ENABLED="true" GO2RTC_HI_RES_ENABLED="true" GO2RTC_HI_RES_ENABLE_AUDIO="true" GO2RTC_LOW_RES_ENABLED="false" GO2RTC_LOW_RES_ENABLE_AUDIO="false" NTP_SERVER="[LOCAL_NTP_SERVER_IP]" DISABLE_FW_UPGRADE="true" ENABLE_SELFHOSTED_MODE="true" WEB_SERVER_ENABLED="false"

the embedded aac encoder from ffmepg has very low latency by default but by far the worst audio quality
- did various tests with VBR and CBR as well as combinations with the available lc profiles
- audio was always unusable due to constant stuttering
- but latency seems to be the same as libopus with 10ms frame duration
libfdkaac delivers perfekt audio quality but only with high latency (nearly 2 seconds)_
- did various tests with the lc profile and different VBR and CBR combinations
- audio quality was very good even down to 16kbps
- even without afterburner the latency was always on the same high level
libopus delivers best overall experience with perfekt audio quality and low latency
- lowest compression_level value doesn’t deliver lowest latency - it is actually the other way around
- the highest value that worked fine with 10ms frame duration on V3 was 7
- 7 has the lowest latency as well as perfect audio quality
- latency increases minimally with each step down
- if I remenber correctly 4 was the lowest value that delievered acceptable audio and latency
- surprisingly 8 increased the latency again by a quite a bit
- and from 9 or with 10 (default if nothing is set) go2rtc crashed in combination with 10ms frame duration
- maybe 7 is close to the edge and gets unstable with parallel low quality stream or on really hot days!?
- I will do a few tests on V2 with T20 soon
- For now I’ll stay with the following setting on V3 which worked best for me without any issues so far

- exec:ffmpeg -hide_banner -f alsa -ac 1 -i dsnooper0 -c:a libopus -compression_level 7 -frame_duration 10 -muxdelay 0 -muxpreload 0 -ac:a 1 -af adelay=0|0 -f rtsp {output}

jemershaw commented 1 year ago

@gtxaspec when you added it to frigate did you use go2rtc in frigate? If so what was the config you used? I tried setting this:

streams:
  camera1:
    - ffmpeg:rtsp://camera1:8554/1080p?video&audio=pcml#video=copy#audio=copy
    - ffmpeg:camera1#video=copy#audio=aac
    - ffmpeg:camera1#video=copy#audio=opus
  camera1.sub:
    - ffmpeg:rtsp://camera1:8554/360p?video&audio=pcml#video=copy#audio=copy
    - ffmpeg:camera1.sub#video=copy#audio=aac
    - ffmpeg:camera1.sub#video=copy#audio=opus

When I try to view it from the frigate server the page just keeps restarting and I don't see a connection. Maybe I'm using the wrong configuration?

UPDATE:

turns out it was a go2rtc problem since ffmpeg didn't support pcml. I changed the cameras to look like this

  camera1:
    - ffmpeg:rtsp://camera1:8554/1080p?video&audio=aac#video=copy#audio=copy
    - ffmpeg:camera1#video=copy#audio=opus

gtxaspec commented 1 year ago

Positive feedback: https://github.com/gtxaspec/wz_mini_hacks/discussions/617#discussioncomment-7039997

gtxaspec commented 1 year ago

@mega3st have 6ou tried with straight pcm audio using copy?

unintended-consequences commented 1 year ago

@mega3st - exec:ffmpeg -hide_banner -f alsa -ac 1 -i dsnooper0 -c:a libopus -compression_level 7 -frame_duration 10 -muxdelay 0 -muxpreload 0 -ac:a 1 -af adelay=0|0 -f rtsp {output}

This is brilliant. Working great for me. Wyzecam v3, Tinycam Pro on Android. For the less technically minded like me:

Loaded the go2rtc webpage at http://[cam-ip]:1984
Click Config
Add # to all other lines starting - exec:ffmpeg -hide_banner...
Uncommented the last '-exec:ffmpeg' line with libopus and updated it to match your settings above
Click Save & Restart
Wait 2 mins to reboot.

KyleStilkey commented 1 year ago

Ok I've had some time to mess with this, I got one camera up and running. It's a v3 but I'm having issues seeing video. Clicking on the stream button for 1080p displays a black screen but I do hear Audio, not sure if I'm missing a something on my end.

pdobrien3 commented 1 year ago

@jemershaw curious if you followed https://github.com/gtxaspec/wz_mini_hacks/issues/550#issuecomment-1663852128 for your wz_mini setup? I can pull streams in VLC but as soon as I limit go2rtc to - exec:ffmpeg -hide_banner -f alsa -ac 1 -i dsnooper0 -c:a libopus -compression_level 7 -frame_duration 10 -muxdelay 0 -muxpreload 0 -ac:a 1 -af adelay=0|0 -f rtsp {output} I cant pull anything. Havent tried in frigate yet but if you have version 3s working?

gtxaspec commented 1 year ago

@pdobrien3 have you checked the go2rtc logs? what do they say when it fails

pdobrien3 commented 1 year ago

So I actually ended up getting a working config. It is late and I am tired so this isn't all inclusive. My wz_mini config is pretty basic. I ended up leaving the go2rtc config at the default and setup Frigate like this:

go2rtc:
  streams:
    1_camera:    
      - ffmpeg:rtsp://192.168.xx.xxx:8554/1080p?video&audio=aac#video=copy#audio=copy
      - ffmpeg:1_camera#video=copy#audio=opus

camera config:

  1_camera:
    ffmpeg:
      hwaccel_args: preset-vaapi
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://192.168.xx.xxx:8554/360p
            - detect
        - path: rtsp://localhost:8554/1_camera
          input_args: preset-rtsp-restream-low-latency
          roles:
            - record

The streams are a little blotchy and I am sure they can be improved upon but, I wanted to just get something up and running. Audio is also a little off sync but really not bad. Will be interesting to see if Frigate makes it through the night without any ffmpeg crashes.

I honestly have no clue what I am doing and feel pretty lucky to get working streams in Frigate. Any advice is greatly appreciated!!!

pdobrien3 commented 1 year ago

woke up to a bunch of this, any idea?

2023-10-22 05:23:06.709167904  [2023-10-22 05:23:06] frigate.video                  ERROR   : 1_camera: Unable to read frames from ffmpeg process.
2023-10-22 05:23:06.709628439  [2023-10-22 05:23:06] frigate.video                  ERROR   : 1_camera: ffmpeg process is not running. exiting capture thread...
2023-10-22 05:23:15.194541440  [2023-10-22 05:23:15] watchdog.1_camera    ERROR   : Ffmpeg process crashed unexpectedly for 1_camera.
2023-10-22 05:23:15.194745284  [2023-10-22 05:23:15] watchdog.1_camera    ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2023-10-22 05:23:15.195005108  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   : [h264 @ 0x5594bcba4ec0] decode_slice_header error
2023-10-22 05:23:15.195229589  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   : [h264 @ 0x5594bcba4ec0] no frame!
2023-10-22 05:23:15.195419995  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   : [h264 @ 0x5594bcba4ec0] non-existing PPS 0 referenced
2023-10-22 05:23:15.195624832  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   :     Last message repeated 1 times
2023-10-22 05:23:15.195832875  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   : [h264 @ 0x5594bcba4ec0] decode_slice_header error
2023-10-22 05:23:15.196025966  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   : [h264 @ 0x5594bcba4ec0] no frame!
2023-10-22 05:23:15.196236635  [2023-10-22 05:23:15] ffmpeg.1_camera.detect ERROR   : [h264 @ 0x5594bcba4ec0] non-existing PPS 0

pdobrien3 commented 1 year ago

so, I changed:

output_args:
        record: preset-record-generic-audio-aac

to:

output_args:
        record: preset-record-generic-audio-copy

I will edit my post above for future visitors.

and frigate has been running rock solid all day. Thanks @gtxaspec for the dev work on go2rtc. I have been struggling with my V3s for a long time. There is some minor audio sync issues but nothing critical and haven't played with any settings. I have all the options in the go2rtc un-hashed, which doesn't jive with this string but things are working.

ajbucci commented 11 months ago

I'm currently using go2rtc with all defaults and am experiencing serious compression artifacts on a wired connection. Does anyone know how to reduce compression/increase bitrate in go2rtc? I presume it could be possible by modifying the go2rtc config?

I didn't realize that the VIDEO settings in the wz_mini.config file also apply to go2rtc... I just bumped up the bitrate there and set to capped quality and now all is good.

mcondren commented 8 months ago

Can anyone share the go2rtc config file line from the wyzecam that sends audio out with raw pcma or pcmu? I saw @gtxaspec mention PCM above but I am unsure what to add to the go2rtc file on the cam to make that happen. For codec issues downstream I need the cams to actually send raw pcm instead of opus or aac. Thanks!

gtxaspec / wz_mini_hacks

feature request: replace v4l2rtspserver with go2rtc+ffmpeg #550

wz_mini configuration

W E B CAMERA

THIS MODE DISABLES EVERYTHING AND IT WILL

WORK AS A WEB CAMERA FOR YOUR PC ONLY

NETWORKING

NETWORK INTERFACE BONDING

ACCESSORIES

VIDEO STREAM

RTMP STREAMING

REQUIRES RTSP

AND RTSP AUDIO

GENERAL

DEBUG

drops you to a shell via serial, doesn't load app_init.sh

WEB

SCRIPTING