Raspberry Pi 5: FFmpeg exited with code: null and signal: SIGSEGV (Unexpected)

[Hulk-Liu]

No more useful logs, everything is working for my thermostat, but both cameras gives the error message constantly.

FFmpeg exited with code: null and signal: SIGSEGV (Unexpected)

Hardware: Raspberry pi 5 vEncoder is default

[potmat]

Try turning on debug mode in Homebridge settings.

This is FFmpeg itself crashing because it's running out of memory or performing some illegal operation. Not sure what's causing it. Did you install ffmpeg yourself? Did it come with your distro? Is it using ffmpeg-for-homebridge?

[Hulk-Liu]

[26/01/2024, 21:07:13] [homebridge-google-nest-sdm] Video stream requested: 1280 x 720, 30 fps, 299 kbps Back door Camera
[26/01/2024, 21:07:13] [homebridge-google-nest-sdm] Executing command sdm.devices.commands.CameraLiveStream.GenerateWebRtcStream with parameters {"offerSdp":"v=0\r\no=- 49056833 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 1 2\r\na=extmap-allow-mixed\r\na=msid-semantic:WMS *\r\nm=audio 9 UDP/TLS/RTP/SAVPF 96\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:1ec6\r\na=ice-pwd:55506dcf54064ea9eb2567\r\na=ice-options:trickle\r\na=fingerprint:sha-256 EE:18:F5:2A:27:3D:E9:8B:36:6F:46:C4:79:38:48:56:40:BA:A2:7D:2B:5B:0F:0A:5A:7E:29:E7:61:A8:70:19\r\na=setup:actpass\r\na=recvonly\r\na=mid:0\r\na=msid:ebc5f7c8-59c8-49f1-90f2-bccc7825b04d 9f7353a9-62b3-43d3-bdfc-1f4b6ce45d97\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=rtcp-mux\r\na=ssrc:3577777056 cname:5b08f3fc-b074-49c6-b952-2e8d0bd32268\r\na=rtpmap:96 opus/48000/2\r\nm=video 9 UDP/TLS/RTP/SAVPF 97\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:1ec6\r\na=ice-pwd:55506dcf54064ea9eb2567\r\na=ice-options:trickle\r\na=fingerprint:sha-256 EE:18:F5:2A:27:3D:E9:8B:36:6F:46:C4:79:38:48:56:40:BA:A2:7D:2B:5B:0F:0A:5A:7E:29:E7:61:A8:70:19\r\na=setup:actpass\r\na=recvonly\r\na=mid:1\r\na=msid:43421174-3b06-4e4c-9d7a-e82b802175ee d8482d6f-b6ae-4f21-b48e-870b3674c01f\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=rtcp-mux\r\na=ssrc:3765355989 cname:5b08f3fc-b074-49c6-b952-2e8d0bd32268\r\na=rtpmap:97 H264/90000\r\na=rtcp-fb:97 transport-cc\r\na=rtcp-fb:97 ccm fir\r\na=rtcp-fb:97 nack\r\na=rtcp-fb:97 nack pli\r\na=rtcp-fb:97 goog-remb\r\na=fmtp:97 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:1ec6\r\na=ice-pwd:55506dcf54064ea9eb2567\r\na=ice-options:trickle\r\na=fingerprint:sha-256 EE:18:F5:2A:27:3D:E9:8B:36:6F:46:C4:79:38:48:56:40:BA:A2:7D:2B:5B:0F:0A:5A:7E:29:E7:61:A8:70:19\r\na=setup:actpass\r\na=mid:2\r\na=sctp-port:5000\r\na=max-message-size:65536\r\n"} Back door Camera
[26/01/2024, 21:07:14] [homebridge-google-nest-sdm] Execution of command sdm.devices.commands.CameraLiveStream.GenerateWebRtcStream returned {"answerSdp":"v=0\r\no=- 0 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 2 1\r\na=msid-semantic: WMS 17887809467961373043/4090014296 virtual-6666\r\na=ice-lite\r\nm=audio 19305 UDP/TLS/RTP/SAVPF 96\r\nc=IN IP4 142.250.111.127\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=candidate: 1 udp 2113939711 2607:f8b0:4023:1401::7f 19305 typ host generation 0\r\na=candidate: 1 tcp 2113939710 2607:f8b0:4023:1401::7f 19305 typ host tcptype passive generation 0\r\na=candidate: 1 ssltcp 2113939709 2607:f8b0:4023:1401::7f 443 typ host generation 0\r\na=candidate: 1 udp 2113932031 142.250.111.127 19305 typ host generation 0\r\na=candidate: 1 tcp 2113932030 142.250.111.127 19305 typ host tcptype passive generation 0\r\na=candidate: 1 ssltcp 2113932029 142.250.111.127 443 typ host generation 0\r\na=ice-ufrag:ciM5fyfIF__8lQoKWigaIAEQABgHCAMGBQQBAggSDAiS\r\na=ice-pwd:KMQTOEUEANYQ//P6ZJDR5PTY\r\na=fingerprint:sha-256 33:90:E3:2B:F6:4E:D4:95:94:C4:41:F1:F7:51:F8:BF:4E:D8:7A:1B:06:ED:77:3A:A4:C4:D8:F4:FF:49:FF:6C\r\na=setup:passive\r\na=mid:0\r\na=sendrecv\r\na=msid:virtual-6666 virtual-6666\r\na=rtcp-mux\r\na=rtpmap:96 opus/48000/2\r\na=fmtp:96 minptime=10;useinbandfec=1\r\na=ssrc:6666 cname:6666\r\nm=video 9 UDP/TLS/RTP/SAVPF 97\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:ciM5fyfIF__8lQoKWigaIAEQABgHCAMGBQQBAggSDAiS\r\na=ice-pwd:KMQTOEUEANYQ//P6ZJDR5PTY\r\na=fingerprint:sha-256 33:90:E3:2B:F6:4E:D4:95:94:C4:41:F1:F7:51:F8:BF:4E:D8:7A:1B:06:ED:77:3A:A4:C4:D8:F4:FF:49:FF:6C\r\na=setup:passive\r\na=mid:1\r\na=sendrecv\r\na=msid:17887809467961373043/4090014296 17887809467961373043/4090014296\r\na=rtcp-mux\r\na=rtpmap:97 H264/90000\r\na=rtcp-fb:97 ccm fir\r\na=rtcp-fb:97 nack\r\na=rtcp-fb:97 nack pli\r\na=rtcp-fb:97 goog-remb\r\na=fmtp:97 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=ssrc:4090014296 cname:4090014296\r\nm=application 9 DTLS/SCTP 5000\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:ciM5fyfIF__8lQoKWigaIAEQABgHCAMGBQQBAggSDAiS\r\na=ice-pwd:KMQTOEUEANYQ//P6ZJDR5PTY\r\na=fingerprint:sha-256 33:90:E3:2B:F6:4E:D4:95:94:C4:41:F1:F7:51:F8:BF:4E:D8:7A:1B:06:ED:77:3A:A4:C4:D8:F4:FF:49:FF:6C\r\na=setup:passive\r\na=mid:2\r\na=sctpmap:5000 webrtc-datachannel 1024\r\n","expiresAt":"2024-01-26T21:12:14.275535Z","mediaSessionId":"ciM5fyfIF__8lQoKWigaIAEQABgHCAMGBQQBAggSDAiS"} Back door Camera
[26/01/2024, 21:07:14] [homebridge-google-nest-sdm] Stream command: ffmpeg -protocol_whitelist pipe,crypto,udp,rtp,fd -analyzeduration 15000000 -probesize 100000000 -i - -an -sn -dn -codec:v libx264 -preset ultrafast -tune zerolatency -pix_fmt yuv420p -color_range mpeg -bf 0 -r 30 -b:v 1196k -bufsize 1196k -maxrate 2392k -filter:v scale='min(1280,iw)':'min(720,ih)':force_original_aspect_ratio=decrease,scale=trunc(iw/2)*2:trunc(ih/2)*2 -payload_type 99 -ssrc 11794389 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params 0tEGLdjQWfkSo2WlEbku3vye0PHCDM+SCJ2BmfEm srtp://192.168.86.56:50988?rtcpport=50988&pkt_size=1378 -vn -sn -dn -codec:a libfdk_aac -profile:a aac_eld -flags +global_header -ar 16k -b:a 24k -ac 1 -payload_type 110 -ssrc 7980079 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params mHU9vzjcqrMY8lWR+DLoRMF72dbd5M1BqS63iLKw srtp://192.168.86.56:63771?rtcpport=63771&pkt_size=188 -loglevel level+verbose v=0
[26/01/2024, 21:07:14] [homebridge-google-nest-sdm] FFmpeg exited with code: null and signal: SIGSEGV (Unexpected) Back door Camera
[26/01/2024, 21:07:22] [homebridge-google-nest-sdm] Executing command sdm.devices.commands.CameraLiveStream.StopWebRtcStream with parameters {"mediaSessionId":"ciM5fyfIF__8lQoKWigaIAEQABgHCAMGBQQBAggSDAiS"} Back door Camera
[26/01/2024, 21:07:24] [homebridge-google-nest-sdm] Execution of command sdm.devices.commands.CameraLiveStream.StopWebRtcStream returned undefined Back door Camera
[26/01/2024, 21:07:24] [homebridge-google-nest-sdm] Stopped video stream. Back door Camera

Did you install ffmpeg yourself? Did it come with your distro? Is it using ffmpeg-for-homebridge?

No I didn't install any ffmpeg and I am using vanilla HB Raspberry pi image. The readme seems implying homebridge-google-nest-sdm will try to install it automatically. Should I manually install ffmpeg?

[potmat]

Well, ffmpeg is there.... the HB RPi image should be fine. For some reason it's getting a segmentation fault and quitting.... I'm afraid from here I really have no idea why.

[JonathanWheeler]

Hey @potmat, thanks for your work and time on this!

I'm having the same issue on my Raspberry Pi 5. I freshly installed Homebridge using the official Raspberry Pi Imager, then immediately installed homebridge-google-nest-sdm as my first plugin, so this is a very clean and fresh install.

Could it be an issue that occurs on Raspberry Pi 5s? Has this been tested on the new Raspberry Pi 5?

Is there anything sensitive in the debug log that I should remove before sharing? Just to give you something immediately, I replaced anything I wasn't sure about with [...]:

[16/02/2024, 01:28:09] [homebridge-google-nest-sdm] Stream command: ffmpeg -analyzeduration 15000000 -probesize 100000000 -i rtsps://stream-us1-bravo.dropcam.com:443/sdm_live_stream/[...] -an -sn -dn -codec:v h264_v4l2m2m -f rawvideo -pix_fmt yuv420p -color_range mpeg -bf 0 -r 30 -b:v 1196k -bufsize 1196k -maxrate 2392k -filter:v scale='min(1280,iw)':'min(720,ih)':force_original_aspect_ratio=decrease,scale=trunc(iw/2)*2:trunc(ih/2)*2 -payload_type 99 -ssrc 3678973 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params [...] srtp://192.168.0.226:59989?rtcpport=59989&pkt_size=1378 -vn -sn -dn -codec:a libfdk_aac -profile:a aac_eld -flags +global_header -ar 16k -b:a 24k -ac 1 -payload_type 110 -ssrc 13953678 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params [...] srtp://192.168.0.226:57236?rtcpport=57236&pkt_size=188 -loglevel level+verbose undefined Living Room Camera
[16/02/2024, 01:28:09] [homebridge-google-nest-sdm] FFmpeg exited with code: null and signal: SIGSEGV (Unexpected) Living Room Camera

I tried using (null), copy, and h264_v4l2m2m for vEncoder. Same result.

It's definitely not running out of memory. Almost all of my 8 GB RAM is free.

Homebridge Raspbian "homebridge/homebridge-raspbian-image-v1.1.6-\(64bit\)" - Raspberry Pi 5 Model B Rev 1.0

Edit 1:

Sorry, it seems you've already determined that it's an upstream ffmpeg issue beyond your control. If there's anything I can do to help troubleshoot, please let me know. Cheers.

Edit 2:

jonathan@Homebridge:~ $ ffmpeg --help
Segmentation fault

Yeah, uh, I guess you aren't going to be fixing it, haha. We may need to loop in FFmpeg and/or Homebridge to fix this.

Edit 3:

I tried sudo apt update && sudo apt upgrade --yes, but that didn't work. (I didn't expect it to, since sudo apt update && apt list --upgradable didn't list ffmpeg, but I was hoping that some system level update might fix things.)

Edit 4:

So I noticed there's a ffmpeg-alpine-arm32v7.tar.gz sitting in the root directory. arm32? 🧐 So I extracted it and compared it to what's in /usr/local/bin. It's a match. I think this is the problem? It's compiled for arm32, not aarch64.

jonathan@Homebridge:~/ffmpeg-alpine-arm32v7/usr/local/bin $ ls -la /ffmpeg-alpine-arm32v7.tar.gz
-rw-r--r-- 1 root root 15989234 Oct  7 11:14 /ffmpeg-alpine-arm32v7.tar.gz
jonathan@Homebridge:~/ffmpeg-alpine-arm32v7/usr/local/bin $ ls -la ffmpeg
-rwxr-xr-x 1 jonathan jonathan 27434160 Oct  7 00:26 ffmpeg
jonathan@Homebridge:~/ffmpeg-alpine-arm32v7/usr/local/bin $ ls -la /usr/local/bin/ffmpeg
-rwxr-xr-x 1 root root 27434160 Oct  7 00:26 /usr/local/bin/ffmpeg
jonathan@Homebridge:~/ffmpeg-alpine-arm32v7/usr/local/bin $ diff ffmpeg /usr/local/bin/ffmpeg

Edit 5:

Tada! Here's the one-liner fix:

sudo curl -Lf# https://github.com/homebridge/ffmpeg-for-homebridge/releases/latest/download/ffmpeg-alpine-$(uname -m).tar.gz | sudo tar xzf - -C / --no-same-owner

Confirmation:

jonathan@Homebridge:~ $ ls -la /usr/local/bin/ffmpeg
-rwxr-xr-x 1 root root 49147296 Oct  7 00:26 /usr/local/bin/ffmpeg
jonathan@Homebridge:~ $ ffmpeg --help
ffmpeg version 6.0-homebridge-alpine-aarch64-static Copyright (c) 2000-2023 the FFmpeg developers
[...]

The video stream indeed now works.

🎉

I'm guessing something isn't installing the correct ffmpeg-for-homebridge binary. The arm32 binary is being installed instead of aarch64. Is this a dependency issue with homebridge-google-nest-sdm, or is this an issue that needs to be forwarded to Homebridge or some other team? If the latter, @potmat, can you please open an issue wherever appropriate referencing this post?

[potmat]

@JonathanWheeler: You nailed it!

Note this from the ffmpeg-for-homebridge readme:

That would seem to be the problem.

As soon as they release an update I'll update here.

[JonathanWheeler]

Thanks, @potmat!

I appreciate what you're saying, but perhaps I'm missing something?

The latest version of ffmpeg-for-homebridge mostly works since it installs the correct architecture, even if there are some misc issues with it.

I tested this myself by reinstalling Homebridge to ensure a clean environment, restored my Homebridge config via backup (which included homebridge-google-nest-sdm, the only package I've installed so far), then performing the following commands:

(Some reboots/restarts may likely be extraneous.)

sudo hb-service update-node
sudo reboot

then:

sudo hb-shell

then:

cd node_modules/homebridge-google-nest-sdm/
npm install ffmpeg-for-homebridge@2.1.1 --save
hb-service rebuild --all
sudo hb-service restart
sudo reboot

This fixed things again, and I didn't notice any other issues.

That said, may we update the ffmpeg-for-homebridge dependency in package.json from 0.1.0 to the latest 2.1.1?

[potmat]

@JonathanWheeler: Good idea I'll update the dependency, give me a day or two.

[potmat]

@JonathanWheeler: Pushed new version 1.2.23 with updated ffmpeg-for-homebridge. This also necessitated updating the NodeJs version.