search

QuantumEntangledAndy / neolink

An RTSP bridge to Reolink IP cameras
GNU Affero General Public License v3.0
330 stars 45 forks source link

MQTT: change spotlight mode (auto/off); turn on/off siren #171

Open chris24walsh opened 11 months ago

chris24walsh commented 11 months ago

I would like to be able to change the spotlight mode (for model Lumus) to enable/disable 'Night Smart Mode', basically it turns on spotlight upon motion during the night time. Also, I would like to be able to trigger the siren.

Can these be added as MQTT entities, just like the IR mode or floodlight switch?

QuantumEntangledAndy commented 11 months ago

As far as I can tell there is no command in the official app to trigger a siren manually (if there is please tell me where). Neolink works like an un-official client so I cannot add new features to the camera only the client.

For the spotlight is that the same as the floodlight? I don't own a spotlight but can you not use /neolink/camname/control/floodlight on?

QuantumEntangledAndy commented 11 months ago

If it is different then you'll need to capture a wireshark dump of manually turning it on/off with the official client so I can check out the command the official client sends to learn how to replicate it

QuantumEntangledAndy commented 11 months ago

Looking through the payloads that the last user who had a flood light sent to me there is this command:

  <?xml version="1.0" encoding="UTF-8" ?>
  <body>
  <FloodlightTask version="1.1">
  <channelId>0</channelId>
  <enable>1</enable> <!-- if any of the motion/night/schedule settings are enabled -->
  <alarmMode>1</alarmMode>
  <preview_auto>0</preview_auto>
  <brightness_cur>92</brightness_cur> <!-- default brightness, or only brightness if just motion mode -->
  <duration>300</duration> <!-- seconds light stays on, at least with motion -->
  <detectType>none</detectType>
  <lastAlarmMode>2</lastAlarmMode>
  <Schedule>
  <startHour>18</startHour>
  <startMin>0</startMin>
  <endHour>6</endHour>
  <endMin>0</endMin>
  </Schedule>
  <lightSensThreshold>
  <lightCur>1000</lightCur>
  <darkCur>1900</darkCur>
  </lightSensThreshold>
  <FloodlightScheduleList />
  <nightLongViewMultiBrightness>  <!-- night mode with usual/alarm brightness -->
  <enable>0</enable>
  <alarmBrightness>
  <cur>100</cur>
  </alarmBrightness>
  <alarmDelay>
  <cur>10</cur> <!-- seconds for alarm brightness -->
  </alarmDelay>
  </nightLongViewMultiBrightness>
  </FloodlightTask>
  </body>

I could try flipping <enable>1</enable> in that but you would have to test it. Since I have no floodlight cameras

chris24walsh commented 11 months ago

Thanks for the reply. There is a button in the official app which turns on/off the siren. It's visible on the live view of the camera, I see it for both Lumus models and also for the Reolink Doorbell model, I've attached some screenshots highlighting it here: lumus doorbell

chris24walsh commented 11 months ago

I could try flipping <enable>1</enable> in that but you would have to test it. Since I have no floodlight cameras

That looks like the switch I'm after, if you could implement it I will test it here.

QuantumEntangledAndy commented 11 months ago

I just got started on working up the xml deserialisation and there are some parts that might need more samples for:

  <nightLongViewMultiBrightness>
  <enable>0</enable>
  <alarmBrightness>
  <cur>100</cur>
  </alarmBrightness>
  <alarmDelay>
  <cur>10</cur> <!-- seconds for alarm brightness -->
  </alarmDelay>
  </nightLongViewMultiBrightness>

and 

<FloodlightScheduleList>
<maxNum>32</maxNum>
</FloodlightScheduleList>

Both seem to turned off in my samples. So I might end up erasing any settings unless I can get more samples. I'll work up the rest but if things get chnged this might be why

QuantumEntangledAndy commented 11 months ago

Ok I added an mqtt command to turn on/off the floodlight tasks. You can test it with this build

chris24walsh commented 11 months ago

Thank you for that! However, it appears the build failed.

Also, regarding the siren button, were you able to determine if this same button exists in your app? I have the iOS app, if that helps. Unfortunately, that makes it harder to capture the network traffic, if you have any suggestions would appreciate them.

QuantumEntangledAndy commented 11 months ago

I found the button also on ios. I've already got package capture setup on my ios so I can get it done, just haven't had the time. Been working on zoom for the PTZ

chris24walsh commented 11 months ago

Thats great, I'll wait for that whenever you can get around to it then. Regarding the build, looks like it failed because it expects a 'floodlight' module, which seems to be undefined.

error[E0583]: file not found for module `floodlight`
  --> crates/core/src/bc_protocol.rs:20:1
   |
20 | mod floodlight;
   | ^^^^^^^^^^^^^^^
   |
   = help: to create the module `floodlight`, create file "crates/core/src/bc_protocol/floodlight.rs" or "crates/core/src/bc_protocol/floodlight/mod.rs"

I notice that the floodlight_status.rs module file was removed but no floodlight.rs file added in that build.

image

QuantumEntangledAndy commented 11 months ago

Yeah I've already fixed it. I renamed the module since it now includes floodlight control as well as status and I forgot to git add -A after the rename.

chris24walsh commented 11 months ago

Figured as much, classic :)

QuantumEntangledAndy commented 11 months ago

If you are still avaliable to test then please try this one

QuantumEntangledAndy commented 11 months ago

Build with siren too is here

chris24walsh commented 11 months ago

Thank you! How do I use the build artifacts produced though? Alternatively, can you create a dev release? I typically run Neolink via the HA addon.

QuantumEntangledAndy commented 11 months ago

The hassio addon is a docker file which pulls latest. The build is avaliable under docker but on a different tag name

You'll need to either find some way to edit the addon to pull the right tag or swap to another way of hosting neolink

chris24walsh commented 11 months ago

My bad, I didn't know that a tag was pushed. I found it in https://hub.docker.com/r/quantumentangledandy/neolink/tags under 'feature-push_mqtt'.

Have tested it by forking and modifying the HA addon to pull from that tag, and updating my neolink.toml with the new 'siren' discovery label - and the siren switch works in HA 🎉 However, the FloodlightTasks switch does not seem operational, for comparison here are the logs from the Mosquitto addon when switching the 'floodlight' switch and the 'floodlighttasks' switch respectively:

2023-10-26 10:18:33: Received PUBLISH from 5ZVf6pVSl8vXy0NC3oOZlP (d0, q0, r0, m0, 'neolink/back_garden/control/floodlight', ... (3 bytes))
2023-10-26 10:18:33: Sending PUBLISH to Neolink (d0, q0, r0, m0, 'neolink/back_garden/control/floodlight', ... (3 bytes))
2023-10-26 10:18:33: Received PUBLISH from Neolink (d0, q1, r0, m61, 'neolink/back_garden/control/floodlight', ... (2 bytes))
2023-10-26 10:18:33: Sending PUBLISH to Neolink (d0, q0, r0, m0, 'neolink/back_garden/control/floodlight', ... (2 bytes))
2023-10-26 10:18:33: Sending PUBACK to Neolink (m61, rc0)
2023-10-26 10:18:33: Received PUBLISH from Neolink (d0, q1, r1, m62, 'neolink/back_garden/status/floodlight', ... (3 bytes))
2023-10-26 10:18:33: Sending PUBLISH to 5ZVf6pVSl8vXy0NC3oOZlP (d0, q0, r0, m0, 'neolink/back_garden/status/floodlight', ... (3 bytes))
2023-10-26 10:18:33: Sending PUBLISH to Neolink (d0, q0, r0, m0, 'neolink/back_garden/status/floodlight', ... (3 bytes))
2023-10-26 10:18:33: Sending PUBACK to Neolink (m62, rc0)
2023-10-26 10:18:33: Received PINGREQ from Neolink
2023-10-26 10:18:33: Sending PINGRESP to Neolink
2023-10-26 10:25:03: Received PUBLISH from 5ZVf6pVSl8vXy0NC3oOZlP (d0, q0, r0, m0, 'neolink/back_garden/control/floodlight_tasks', ... (2 bytes))
2023-10-26 10:25:03: Sending PUBLISH to Neolink (d0, q0, r0, m0, 'neolink/back_garden/control/floodlight_tasks', ... (2 bytes))
2023-10-26 10:25:03: Received PINGREQ from Neolink
2023-10-26 10:25:03: Sending PINGRESP to Neolink
chris24walsh commented 11 months ago

Unfortunately, the camera streams no longer seem to work with this tag. I also get this error showing up in the logs:

[2023-10-26T10:21:33Z WARN neolink::common::pushnoti] Issue connecting to push notifications server: ServerError("PHONE_REGISTRATION_ERROR")

QuantumEntangledAndy commented 11 months ago

The push notification registration should not really matter, it is used for waking up neolink when push alarms are sent.

What is the issue in your client, anything there? Perhaps you can tell me what you mean by no longer seem to work? No entitity in HA, black stream, static stream, no video, any errors in any logs at all. The statement no longer seem to work is not particularly helpful for tracking down an issue in 1000s of lines of code. I will see if I can replicate but my HA is setup with docker compose and dosen't use integrations

chris24walsh commented 11 months ago

Sorry for being vague, just mentioned in case their was some known issue with this branch for the rtsp stream. There are no error messages in the Neolink logs. Connections just fail due to timeout, both for go2rtc and when trying directly with e.g. VLC client. I get this whether I use the 'rtsp' or 'dual' configuration. Here is what is shown in go2rtc logs:

2023-10-27 08:46:02.660864431 09:46:02.660 WRN [rtsp] error="read tcp 172.30.33.14:33232->192.168.2.8:8557: i/o timeout,read tcp 172.30.33.14:37672->192.168.2.8:8557: i/o timeout" stream=back_garden

I'm using go2rtc version 1.6.2, installed via Frigate addon. Never had this before with Neolink's master branch. The logs from Neolink show that the stream has started:

[2023-10-27T08:45:11Z INFO  neolink::rtsp::stream] back_garden: Avaliable at /back_garden/main, /back_garden/Main, /back_garden/mainStream, /back_garden/MainStream, /back_garden/Mainstream, /back_garden/mainstream, /back_garden
[2023-10-27T08:45:11Z INFO  neolink::rtsp::stream] back_garden: Avaliable at /back_garden/sub, /back_garden/Sub, /back_garden/subStream, /back_garden/SubStream, /back_garden/Substream, /back_garden/substream

What is strange though, is that I have stream = "subStream" set in my neolink.toml config file, so I should only see the subStream started.

QuantumEntangledAndy commented 11 months ago

Don't suppose you can show your config. You may have the stream = "subStream" in after a sub table

QuantumEntangledAndy commented 11 months ago

Is it possible to set the environmental variable RUST_LOG="neolink=debug" for the neolink container? It will give more full logs maybe can see where the timeout is happening

chris24walsh commented 11 months ago

Don't suppose you can show your config. You may have the stream = "subStream" in after a sub table

My fault here, I had reverted to an older version of the config file, and was not using the stream = setting. Have fixed that, and no longer see both streams starting, but still have timeouts when trying to connect. Here is my (abridged & redacted) current config:

...
...
bind = "0.0.0.0"
...
...
[mqtt]
broker_addr = "192.168.2.8" # Address of the mqtt server
port = 1883 # mqtt servers port
credentials = ["mqtt", "<redacted>"]
...
...
[[cameras]]
name = "back_garden"
username = "admin"
password = "<reacted>"
address = "192.168.2.14:9000"
stream = "subStream"
  [cameras.mqtt]
    enable_preview = false
    [cameras.mqtt.discovery]
    topic = "homeassistant"
    features = ["floodlight", "ir", "motion", "reboot", "siren"]
...
...

Is it possible to set the environmental variable RUST_LOG="neolink=debug" for the neolink container? It will give more full logs maybe can see where the timeout is happening

Was able to set this env var in the Dockerfile, and grabbed these hopefully relevant logs:

[2023-10-27T10:17:34Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for Watchdog
[2023-10-27T10:17:34Z DEBUG neolink::common::streamthread] back_garden::subStream:   Got frame
[2023-10-27T10:17:34Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for frame
[2023-10-27T10:17:34Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for Watchdog
[2023-10-27T10:17:34Z DEBUG neolink::common::streamthread] back_garden::subStream:   Got frame
[2023-10-27T10:17:34Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for frame
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::stream] StreamConfig { resolution: [704, 480], vid_format: H264, aud_format: Aac, bitrate: 393216, fps: 10 }
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::stream] back_garden: Waiting for Valid Audio
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::stream] StreamConfig { resolution: [704, 480], vid_format: H264, aud_format: Aac, bitrate: 393216, fps: 10 }
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::stream] back_garden: Creating Client Counters
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::stream] back_garden: Stream Activated
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::gst::factory] Constructing Factor Impl
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::gst::factory] Adding anonymous as permitted user
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::stream] Path: /back_garden/sub
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::stream] Path: /back_garden/Sub
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::stream] Path: /back_garden/subStream
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::stream] Path: /back_garden/SubStream
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::stream] Path: /back_garden/Substream
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::stream] Path: /back_garden/substream
[2023-10-27T10:17:34Z DEBUG neolink::rtsp::stream] Path: /back_garden
[2023-10-27T10:17:34Z INFO  neolink::rtsp::stream] back_garden: Avaliable at /back_garden/sub, /back_garden/Sub, /back_garden/subStream, /back_garden/SubStream, /back_garden/Substream, /back_garden/substream, /back_garden

Please let me know what to look for in terms of more relevant logging here.

QuantumEntangledAndy commented 11 months ago

Do you see anything about a media being created. It looks like no connection is being made so maybe this is a docker network thing

chris24walsh commented 11 months ago

I see these kinds of logs repeated over and over:

[2023-10-27T10:39:04Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for Watchdog
[2023-10-27T10:39:04Z DEBUG neolink::common::streamthread] back_garden::subStream:   Got frame
[2023-10-27T10:39:04Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for frame
[2023-10-27T10:39:04Z DEBUG neolink_core::bc_protocol::connection::bcsub] PayloadStream got binary data: 3
[2023-10-27T10:39:04Z DEBUG neolink_core::bc_protocol::connection::bcsub] - BcMedia Payloader got packet
[2023-10-27T10:39:04Z DEBUG neolink_core::bc_protocol::connection::bcsub]   - BcMedia Payloader decoded packet
chris24walsh commented 11 months ago

And:

[2023-10-27T10:40:53Z DEBUG neolink::rtsp::stream] New media

QuantumEntangledAndy commented 11 months ago

Can you show what's in between

"New media"

And

"Vid Thread End"

(Or just post the whole log and I'll spoiler tag it)

QuantumEntangledAndy commented 11 months ago

I see these kinds of logs repeated over and over:


[2023-10-27T10:39:04Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for Watchdog

[2023-10-27T10:39:04Z DEBUG neolink::common::streamthread] back_garden::subStream:   Got frame

[2023-10-27T10:39:04Z DEBUG neolink::common::streamthread] back_garden::subStream:   Waiting for frame

[2023-10-27T10:39:04Z DEBUG neolink_core::bc_protocol::connection::bcsub] PayloadStream got binary data: 3

[2023-10-27T10:39:04Z DEBUG neolink_core::bc_protocol::connection::bcsub] - BcMedia Payloader got packet

[2023-10-27T10:39:04Z DEBUG neolink_core::bc_protocol::connection::bcsub]   - BcMedia Payloader decoded packet

I was recently debugging how long between packets from the camera. Every time you get one of these a new media packet is received from the camera.

QuantumEntangledAndy commented 11 months ago

binary data: 3 is a media packet. Several of these are concatenated together to make a BCMedia frame which is usually a whole video IFrame/PFrame or an AAC packet.

QuantumEntangledAndy commented 11 months ago

Do yo have anything about watchdog kicking the stream? Any time that happens it means no frames for 10s

chris24walsh commented 11 months ago

I don't find 'Video Thread End' showing up in the logs at all. Here is a gist showing a snippet between two 'New media' entries (too many lines for github comment):

https://gist.github.com/chris24walsh/09cd2e97f7fa212c34d7b0fd057031b3

Btw, this shows several camera entries, didn't want to attempt to prune it down to one camera.

chris24walsh commented 11 months ago

Do yo have anything about watchdog kicking the stream? Any time that happens it means no frames for 10s

Only thing I see for Watchdog is Waiting for Watchdog.

chris24walsh commented 11 months ago

Here is the same excerpt, filtered on 'rtsp':

[2023-10-27T11:47:36Z DEBUG neolink::rtsp::stream] New media
[2023-10-27T11:47:36Z DEBUG neolink::rtsp::stream] stream_config.fps: 10
[2023-10-27T11:47:36Z DEBUG neolink::rtsp::stream] back_garden: Number of rtsp clients: 3
[2023-10-27T11:47:36Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:36Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:36Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:37Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:37Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:37Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:37Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:37Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:37Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:37Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:38Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:38Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:39Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:39Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:39Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:40Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:40Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:40Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:40Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:40Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:40Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:40Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:40Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:41Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:41Z DEBUG neolink::rtsp::stream] Key Frame: Resume
[2023-10-27T11:47:41Z DEBUG neolink::rtsp::factory] Clearing old elements
[2023-10-27T11:47:41Z DEBUG neolink::rtsp::factory] buffer_size: 4194304
[2023-10-27T11:47:41Z DEBUG neolink::rtsp::factory] Building H264 Pipeline
[2023-10-27T11:47:41Z DEBUG neolink::rtsp::factory] buffer_size: 4194304
[2023-10-27T11:47:41Z DEBUG neolink::rtsp::factory] Building Aac pipeline
[2023-10-27T11:47:41Z DEBUG neolink::rtsp::stream] New media

One thing I notice, is the increasing 'Number of rtsp clients' for each camera over time. Is that normal? (Shown here after container restarted btw)

QuantumEntangledAndy commented 11 months ago

Kind of normal. Some clients (go2rc for example) will not cleanly close an rtsp stream on error. I recently reduced the timeout for clients to 5s to try to combat this, but no user who uses frigate has had the time to report back to me if this is actually helping

QuantumEntangledAndy commented 11 months ago

The logs for "Cleaned up {cleanups} sessions" reports how many were cleaned up and "random_chars: number/5" should show how many ms left to hangup

QuantumEntangledAndy commented 11 months ago

Trying to get my frigate up but some issues on the rpi it is running on. Might need to re-format the SD card

chris24walsh commented 11 months ago

Any chance you could merge the 'siren' code to master? Its the functionality I'm most eager for. It seems to work fine, not sure if it requires other stuff from this feature branch?

I might try to create a PR for that, save you the hassle.

chris24walsh commented 11 months ago

https://github.com/QuantumEntangledAndy/neolink/pull/175

chris24walsh commented 11 months ago

Unfortunately, the new 0.6.3-rc.1 release means that the latest tag is now broken for me, since the stream is not working. Must be something to do with using the arm64 build, since I'm running Neolink via the HA addon on a HA Blue (Odroid N2). I'll have to stick with my fork of the addon so i can stay on the 0.6.2 tag. Thanks anyway for the work put it for the siren/floodlight stuff. Hope you find a fix for the streams on my build soon. And also the floodlight_tasks switch (doesn't work for me).