search

QuantumEntangledAndy / neolink

An RTSP bridge to Reolink IP cameras
GNU Affero General Public License v3.0
330 stars 45 forks source link

Feature/add siren #175

Closed chris24walsh closed 11 months ago

chris24walsh commented 11 months ago

Would be nice to have the benefit of your work adding control of the siren, until your feature/push_mqtt branch is ready.

QuantumEntangledAndy commented 11 months ago

I think feature/push_mqtt is already in a good enough state, I can get the stream to work in VLC, HA, Frigate with and without go2rc. The branch also fixes a ton of issues with 0.6.2 so I will release together

chris24walsh commented 11 months ago

Ok, cool. I'll wait for that to be merged. Any rough ETA?

QuantumEntangledAndy commented 11 months ago

Already merged and released as an rc

chris24walsh commented 11 months ago

Ok, unfortunately this means that the latest tag is now broken for me, since the streams are not working. Must be something to do with using the arm64 build, since I'm running Neolink via the HA addon on a HA Blue (Odroid N2). I'll have to stick with my fork of the addon so i can stay on the 0.6.2 tag. Thanks anyway for the work put it for the siren/floodlight stuff. Hope you find a fix for the streams on my build soon. And also the floodlight_tasks switch (doesn't work for me).

QuantumEntangledAndy commented 11 months ago

Not the arm64 since my frigate tests are all done on arm64 on my rpi.

chris24walsh commented 11 months ago

Huh, I'm at a loss then. The docker container running for the addon on my HA shows as 'aarch64-addon-neolink'. Guess that is arm64.

QuantumEntangledAndy commented 11 months ago

What are the resolution and bitrates of your streams? If you put them all on minimum does that work? Perhaps there's something up with the buffering that needs tweaking.

chris24walsh commented 11 months ago

They were already close to minimum, since I found that helped before to prevent connection drops. I put it all the way down now to the minimum bitrate of 128kbps, and resolution is fixed at 704x480. Still no good.

QuantumEntangledAndy commented 11 months ago

Oh well worth a try. When you do that does the stream say h264 or h265 in the debug logs?

chris24walsh commented 11 months ago

H264. I got some more details from the logs, showing what you asked for before, 'Vid Thread End'. Doesn't look too good:

2023-10-30T14:45:47Z DEBUG neolink::rtsp::factory] Clearing old elements
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::factory] buffer_size: 4194304
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::factory] Building H264 Pipeline
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::factory] buffer_size: 4194304
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::factory] Building Aac pipeline
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] New media
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] stream_config.fps: 10
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] back_garden: Number of rtsp clients: 6
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] Vid Thread End: Err(App source is closed)
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] back_garden: Number of rtsp clients: 5
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] Aud Thread End: Err(App source is closed)
[2023-10-30T14:45:47Z DEBUG neolink_core::bc_protocol::connection::bcsub] PayloadStream got binary data: 3
[2023-10-30T14:45:47Z DEBUG neolink_core::bc_protocol::connection::bcsub] - BcMedia Payloader got packet
[2023-10-30T14:45:47Z DEBUG neolink_core::bc_protocol::connection::bcsub]   - BcMedia Payloader decoded packet
[2023-10-30T14:45:47Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for Watchdog
[2023-10-30T14:45:47Z DEBUG neolink::common::streamthread] back_garden::subStream:   Got frame
[2023-10-30T14:45:47Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for frame
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] Aud Thread End: Err(App source is closed)
[2023-10-30T14:45:47Z DEBUG neolink_core::bc_protocol::connection::bcsub] PayloadStream got binary data: 3
[2023-10-30T14:45:47Z DEBUG neolink_core::bc_protocol::connection::bcsub] - BcMedia Payloader got packet
[2023-10-30T14:45:47Z DEBUG neolink_core::bc_protocol::connection::bcsub]   - BcMedia Payloader decoded packet
[2023-10-30T14:45:47Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for Watchdog
[2023-10-30T14:45:47Z DEBUG neolink::common::streamthread] back_garden::subStream:   Got frame
[2023-10-30T14:45:47Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for frame
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] Aud Thread End: Err(App source is closed)
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] Vid Thread End: Err(App source is closed)
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] back_garden: Number of rtsp clients: 4
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] Aud Thread End: Err(App source is closed)
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] Vid Thread End: Err(App source is closed)
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] back_garden: Number of rtsp clients: 3
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] Vid Thread End: Err(App source is closed)
[2023-10-30T14:45:47Z DEBUG neolink::rtsp::stream] back_garden: Number of rtsp clients: 2
[2023-10-30T14:45:47Z DEBUG neolink_core::bc_protocol::connection::bcsub] PayloadStream got binary data: 3
[2023-10-30T14:45:47Z DEBUG neolink_core::bc_protocol::connection::bcsub] - BcMedia Payloader got packet
[2023-10-30T14:45:47Z DEBUG neolink_core::bc_protocol::connection::bcsub]   - BcMedia Payloader decoded packet
[2023-10-30T14:45:47Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for Watchdog
chris24walsh commented 11 months ago

Ok, something new. Since I removed all the other cameras from the Neolink config, and stopped Frigate (so no connections from there), I can open the stream from Neolink for the single camera configured on IPCam mobile app. Seems indeed that it works, just struggles in the latest version to process the stream/connections. It's still very slow to connect, and prone to dropping. Here are the logs when starting Neolink, btw:

--- VERSIONS ---
add-on version: 0.0.8
[2023-10-30T15:09:13Z INFO  neolink] Neolink 4a94a2ddb29bed207307eb2de744748190e8577f release
neolink version: neolink 0.6.3-rc.1
neolink mode: rtsp
neolink log: debug
ATTENTION: if you expected a newer Neolink version, please reinstall this Add-on!
--- Neolink ---
[2023-10-30T15:09:13Z INFO  neolink] Neolink 4a94a2ddb29bed207307eb2de744748190e8577f release
[2023-10-30T15:09:13Z INFO  neolink::rtsp] Starting RTSP Server at 0.0.0.0:8554
[2023-10-30T15:09:13Z INFO  neolink::rtsp] back_garden: Rtsp Staring
[2023-10-30T15:09:13Z DEBUG neolink::common::reactor] Inserting new insance
[2023-10-30T15:09:13Z INFO  neolink::utils] back_garden: Connecting to camera at Address: 192.168.2.14:9000
[2023-10-30T15:09:13Z INFO  neolink_core::bc_protocol] back_garden: Trying TCP discovery
[2023-10-30T15:09:13Z DEBUG neolink::common::reactor] New instance created
[2023-10-30T15:09:13Z DEBUG neolink::common::reactor] Got instance from reactor
[2023-10-30T15:09:13Z DEBUG neolink::rtsp] back_garden: Camera Main
[2023-10-30T15:09:13Z DEBUG neolink::rtsp] back_garden: Camera Main::Loop
[2023-10-30T15:09:13Z DEBUG neolink::rtsp] back_garden: Camera Main::Select Stream
[2023-10-30T15:09:13Z DEBUG neolink::rtsp::gst::factory] Constructing Factor Impl
[2023-10-30T15:09:13Z DEBUG neolink::rtsp::gst::factory] Adding anonymous as permitted user
[2023-10-30T15:09:13Z DEBUG neolink::rtsp] back_garden: Camera Main::Select Sub
[2023-10-30T15:09:13Z DEBUG neolink::rtsp] Path: /back_garden/sub
[2023-10-30T15:09:13Z DEBUG neolink::rtsp] Path: /back_garden/Sub
[2023-10-30T15:09:13Z DEBUG neolink::rtsp] Path: /back_garden/subStream
[2023-10-30T15:09:13Z DEBUG neolink::rtsp] Path: /back_garden/SubStream
[2023-10-30T15:09:13Z DEBUG neolink::rtsp] Path: /back_garden/Substream
[2023-10-30T15:09:13Z DEBUG neolink::rtsp] Path: /back_garden/substream
[2023-10-30T15:09:13Z DEBUG neolink::rtsp] Path: /back_garden
[2023-10-30T15:09:13Z DEBUG neolink::rtsp] back_garden: Preparing at /back_garden/sub, /back_garden/Sub, /back_garden/subStream, /back_garden/SubStream, /back_garden/Substream, /back_garden/substream, /back_garden
[2023-10-30T15:09:13Z INFO  neolink_core::bc_protocol] back_garden: TCP Discovery success at 192.168.2.14:9000
[2023-10-30T15:09:13Z INFO  neolink::utils] back_garden: Logging in
[2023-10-30T15:09:13Z DEBUG neolink_core::bc_protocol::login] Populating abilities
[2023-10-30T15:09:13Z DEBUG neolink_core::bc_protocol::connection::bcconn] Ignoring uninteresting message id 78 (number: 9)
[2023-10-30T15:09:13Z DEBUG neolink_core::bc_protocol::connection::bcconn] Ignoring uninteresting message id 291 (number: 0)
[2023-10-30T15:09:13Z DEBUG neolink_core::bc_protocol::abilityinfo] Abilities: <?xml version="1.0" encoding="utf-8"?><AbilityInfo><userName>admin</userName><system><subModule><abilityValue>general_rw, norm_rw, version_ro, uid_ro, autoReboot_rw, restore_rw, reboot_rw, shutdown_rw, dst_rw, log_ro, performance_ro, upgrade_rw, export_rw, import_rw, bootPwd_rw</abilityValue></subModule></system><network><subModule><abilityValue>port_rw, dns_rw, email_rw, ipFilter_rw, localLink_rw, pppoe_rw, upnp_rw, wifi_rw, ntp_rw, netStatus_rw, ptop_rw, autontp_rw</abilityValue></subModule></network><alarm><subModule><channelId>0</channelId><abilityValue>motion_rw</abilityValue></subModule></alarm><image><subModule><channelId>0</channelId><abilityValue>ispBasic_rw, ispAdvance_rw, ledState_rw</abilityValue></subModule></image><video><subModule><channelId>0</channelId><abilityValue>osdName_rw, osdTime_rw, shelter_rw</abilityValue></subModule></video><security><subModule><abilityValue>user_rw, userOnline_rw, bootPwd_rw</abilityValue></subModule></security><PTZ><subModule><abilityValue>control_rw, preset_rw, cruise_rw, track_rw, decoder_rw, ptzInfo_ro</abilityValue></subModule></PTZ><streaming><subModule><channelId>0</channelId><abilityValue>preview_rw, compress_rw, snap_rw, rtsp_rw, streamTable_ro</abilityValue></subModule></streaming></AbilityInfo>
[2023-10-30T15:09:13Z INFO  neolink::utils] back_garden: Connected and logged in
[2023-10-30T15:09:15Z INFO  neolink::common::camthread] back_garden: Camera time is already set: 2023-10-30 15:09:16.0 +00:00:00
[2023-10-30T15:09:16Z DEBUG neolink::common::pushnoti] Push notification details are saved to Some("/root/.config/./neolink_token.toml")
[2023-10-30T15:09:16Z DEBUG neolink::common::pushnoti] Registering new push notification token
[2023-10-30T15:09:17Z INFO  neolink::common::neocam] back_garden: Model Reolink Lumus
[2023-10-30T15:09:17Z INFO  neolink::common::neocam] back_garden: Firmware Version v2.0.0.705_21052800
[2023-10-30T15:09:17Z DEBUG neolink::common::neocam] back_garden: Listing Camera Supported Streams
        mainStream: 1920x1080
        subStream: 704x480
[2023-10-30T15:09:17Z DEBUG neolink::common::streamthread] back_garden::subStream: Running Stream Instance Task
[2023-10-30T15:09:17Z DEBUG neolink::rtsp::stream] back_garden: Activating Stream
[2023-10-30T15:09:17Z DEBUG neolink::rtsp::stream] back_garden: Waiting for Valid Stream
[2023-10-30T15:09:23Z DEBUG neolink::rtsp::factory] Clearing old elements
[2023-10-30T15:09:23Z DEBUG neolink::rtsp::factory] buffer_size: 4194304
[2023-10-30T15:09:23Z DEBUG neolink::rtsp::factory] Building H264 Pipeline
[2023-10-30T15:09:23Z DEBUG neolink::rtsp::factory] buffer_size: 4194304
[2023-10-30T15:09:23Z DEBUG neolink::rtsp::factory] Building Aac pipeline
[2023-10-30T15:09:23Z DEBUG neolink::rtsp::stream] New media
[2023-10-30T15:09:23Z DEBUG neolink::rtsp::stream] stream_config.fps: 10
[2023-10-30T15:09:23Z DEBUG neolink::rtsp::stream] back_garden: Number of rtsp clients: 1

P.S. I updated the logs, since I got a greater number of relevant log lines.

chris24walsh commented 11 months ago

Hmmm, I don't seem to have audio though. Is that relevant? (I do for this app client when using the older version of Neolink)

chris24walsh commented 11 months ago

Info from the app client:

chris24walsh commented 11 months ago

Adding another camera to Neolink's config fails, the additional camera won't load, and the first struggles even more to establish a connection (buffering/loading takes longer).

chris24walsh commented 11 months ago

For comparison, on version 0.6.2 I was able to load 5 streams (all Lumus cameras) without too much trouble. I did have to lower the bitrates (to 256) to make it work. And there were still some sporadic errors and 'rtsp killed' crashes in Neolink. I would have to restart periodically (every few days it seemed). But all 5 were passed into Frigate, so were persistent connections.

QuantumEntangledAndy commented 11 months ago

This is quite odd how you get such a difference between the two. If anything I expected more streams since optimisations were performed to reduced both the memory consumption and cpu usage

Do you notice any significant change in the cpu or ram between the two?

chris24walsh commented 11 months ago

They seem to be about the same cpu/ram usage-wise.

QuantumEntangledAndy commented 11 months ago

Then this will likely be hard for me to diagnose, since I am not observing it and cannot devote this much time to tracing the error remotly

chris24walsh commented 11 months ago

I understand, sorry I can't provide more useful details. Hopefully, I can identify something more pertinent in the future.

chris24walsh commented 11 months ago

One thing I've noticed going back and forth between 0.6.2 and 0.6.3 is there is a huge difference in loading/buffering time when opening a stream; ~5 seconds vs. >30 seconds respectively. (Note that I can only open the Neolink streams via the third party IPCam iOS app, so I'm comparing the loading/buffering times using that app for both Neolink versions). Also, I tested the latest 0.6.3 rc on another friends HA installation, albeit with just one Reolink Lumus. But it seems to work ok for him, including passing the stream into Frigate via native go2rtc restream. Only thing is, as mentioned, it takes ages for the stream to begin working whenever restarting Frigate or Neolink. Could this be a clue to the root cause for why it won't work for me?

P.S. Forgot to mention earlier, I am using a different installed version of go2rtc (1.6.2) than included in the latest Frigate addon (1.2.0), which is also a difference between my and my friends setup (need v1.6.2 for RTSP backchannel/2-way audio support for my Reolink Doorbell).

chris24walsh commented 11 months ago

And if we can't solve this, is there some way you could checkout a new feature branch in your repo from my forked branch, so I can benefit from the tagged build? Last time I tried to build my own Neolink image, I had a lot of trouble. Would just like to have an image to pull in my forked HA addon.

QuantumEntangledAndy commented 11 months ago

The buffering is actually just because I haven't added the buffer size control to the config yet. I swapped the buffer internal working from frames to time and left it at a large value to work with my spotty 3G network during my tests. Should add that control back in.

The buffer though does not need to be filled before a connection begins. If you start the stream early while there is only 2s of stream it will play with just a 2s buffer.

I can try some things with the buffer to dump it all into the feed rather than spoon feeding it though the pipeline but need a way to test it easily.

chris24walsh commented 11 months ago

Ok, looking forward to seeing if this fixes anything for my setup.

chris24walsh commented 11 months ago

Solved! Rolling back to go2rtc version 1.2.0 (Frigate default) fixed the streams. I can see that they still take longer to connect, but I guess they are not timing out any longer. Perhaps you can now reproduce to identify the fix required for v1.6.2. Frigate v0.13.0 should be out soon, and I understand that it will use >1.6.2, so this will surely affect more users shortly.

chris24walsh commented 11 months ago

In fact, looks like next Frigate release will use v1.8.1 of go2rtc. I've tried 1.8.2 here, streams still don't work with Neolink v0.6.3-rc.1. Seems compatibility broke between 1.2.0 and 1.6.2 somewhere.

chris24walsh commented 10 months ago

Seems others also using the latest 0.6.3 with Reolink Lumus and Frigate are encountering relevant issues:

https://github.com/QuantumEntangledAndy/neolink/issues/183 https://github.com/QuantumEntangledAndy/neolink/issues/188

They haven't mentioned what version of go2rtc they are using though, wonder if they are able to use >=1.6.2

chris24walsh commented 6 months ago

Solved! Rolling back to go2rtc version 1.2.0 (Frigate default) fixed the streams. I can see that they still take longer to connect, but I guess they are not timing out any longer. Perhaps you can now reproduce to identify the fix required for v1.6.2. Frigate v0.13.0 should be out soon, and I understand that it will use >1.6.2, so this will surely affect more users shortly.

Have you managed to have a look at this @QuantumEntangledAndy?

chris24walsh commented 6 days ago

Solved! Rolling back to go2rtc version 1.2.0 (Frigate default) fixed the streams. I can see that they still take longer to connect, but I guess they are not timing out any longer. Perhaps you can now reproduce to identify the fix required for v1.6.2. Frigate v0.13.0 should be out soon, and I understand that it will use >1.6.2, so this will surely affect more users shortly.

Have you managed to have a look at this @QuantumEntangledAndy?

@QuantumEntangledAndy is this on your radar, should I open a new issue for it?