search

thirtythreeforty / neolink

An RTSP bridge to Reolink IP cameras
https://www.thirtythreeforty.net/posts/2020/05/hacking-reolink-cameras-for-fun-and-profit/
GNU Affero General Public License v3.0
875 stars 143 forks source link

Use camera's motion detection #58

Open QuantumEntangledAndy opened 3 years ago

QuantumEntangledAndy commented 3 years ago

Some camera's come with motion detection. I would like to have some way for neolink to signal that motion was detected.

For now I think just printing to std-out but ultimately I am thinking of some sort of control interface (maybe a rest API) would be good. I envisage we can add command and control to this control interface, PTZ turning the camera on and off etc etc.

Here are the relevent wireshark packets for motion detection

motion.pcapng.zip

It is just a BC packet of the msg-id 33:

thirtythreeforty commented 3 years ago

Agree, it's good to have a tracking issue.

The industry standard here is ONVIF, and if Neolink were an ONVIF bridge you could connect any client to it and it would "just work" with all these more advanced features. ONVIF is also somewhat long in the tooth, and I don't see many open source libraries for being an ONVIF server (except one in Javascript of all things).

Ideas for other standard APIs are welcome, but I'd like to avoid rolling something custom for Neolink - it'll just make work for anyone who wants to consume it.

QuantumEntangledAndy commented 3 years ago

Yes I actually wanted ONVIF but as you say I too couldn't find a server version.

Riza-Aslan commented 3 years ago

Could this be helpful to you Guys?

https://github.com/EliasKotlyar/Xiaomi-Dafang-Hacks/issues/78

QuantumEntangledAndy commented 3 years ago

I had a look at their PR that added it but it seems to be a binary commit so I can't read the source and the changes. Some of the links in that issue may be useful though.

thirtythreeforty commented 3 years ago

The actual code is in Dafang-Hacks/Main#41. Their overall approach is to use Gsoap to generate bindings to the ONVIF spec.

QuantumEntangledAndy commented 3 years ago

Thanks I've looked at that now my c++ is rusty (no pun intended) but it seems somewhat doable but quite some work.

It would be best if we could find a soap implementation in rust ive found this but it seems quite minimal. But perhaps a good starting point. It will take soap scheme and generate code. It's designed for a client though not a server.

We could alternatively generate c code with gsoap then either bind or port it.

QuantumEntangledAndy commented 3 years ago

This is promising here it's a soap schema->rust built with ONVIF in mind

QuantumEntangledAndy commented 3 years ago

This may also be of use in generating from soap

QuantumEntangledAndy commented 3 years ago

This is a simple server and client written in C++ but will probably be helpful

digiblur commented 3 years ago

It would be pretty slick if it would output motion events via MQTT.

QuantumEntangledAndy commented 3 years ago

MQTT might be easier to implement than ONVIF too should be able to connect to home assistant as well.

I plan to add ONVIF when I have time but will also look into MQTT too

digiblur commented 3 years ago

Great... I've been bugging manufacturers over and over about some simple MQTT publishes and subscribes. Would be awesome for doing integrations with NodeRed, HomeAssistant, or whatever.

QuantumEntangledAndy commented 3 years ago

I'm thinking to make the MQTT messages complimant enough to work with this HA plugin and use this rust library for the server side code

QuantumEntangledAndy commented 3 years ago

As I have understand the plugin we set the topic to the cameraname the subtopic to motion and the message as either motion_start motion_stop would that work with the design in nodered you had?

digiblur commented 3 years ago

I don't have the plugin. Pretty wide open to however it would be implemented. A good idea might be to do a prefix configuration. Maybe something like //state/motion then a payload of ON or OFF.

neolink/reardoor/state/motion with a payload on or off

Thinking ahead where other stuff could be output as states, like LWT, etc. Then if it is possible, the ability to send commands to the camera like to turn on the white LEDs on the Lumus (not sure if that is possible)

send an on payload to neolink/reardoor/command/light

Might a bit much but you get the idea.

QuantumEntangledAndy commented 3 years ago

I don't mind but this would be going down the route of rolling our own interface, it might be better if there were a standard but it seems from most mqtt projects I have seen that it is more transport than protocol and they often roll there own messages

At least on the plus side the mqtt clients like nodered and HA are configurable in which message to listen for

QuantumEntangledAndy commented 3 years ago

I do eventually plan to get controls set up too but thats a longer project

QuantumEntangledAndy commented 3 years ago

@thirtythreeforty Could I get some recommendations on where to implement the subscribe for msg_id 33 (motion)

Currently we subscribe to id 3 in the start_video function. Then we loop polling the rx with a timeout. Do you think I should add a second subscription here in the loop polling id 33 as well?

QuantumEntangledAndy commented 3 years ago

@digiblur I think I have a working implementation in #78 would you be willing to test it?

digiblur commented 3 years ago

@digiblur I think I have a working implementation in #78 would you be willing to test it?

Sure!

QuantumEntangledAndy commented 3 years ago

Cool, could you checkout the last version from my github assets here

Then add this to your neolink.toml

[[mqtt.servers]]
port = 8883
next_connection_delay_ms = 10
connection_timeout_ms = 100
max_client_id_len = 256
max_connections = 1
throttle_delay_ms = 0
disk_persistence = false
disk_retention_size = 100
disk_retention_time_sec = 1000
auto_save_interval_sec = 1000
max_payload_size = 2048
max_inflight_count = 100
max_inflight_size = 1024
instant_ack = false
# this enables tls connection
#cert_path = "tlsfiles/server.cert.pem"
#key_path = "tlsfiles/server.key.pem"
# provide ca_path to enable client authentication
#ca_path = "tlsfiles/ca-chain.cert.pem"

[mqtt.router]
id = 0
dir = "/tmp/rumqttd"
# A commitlog read will pull full segment. Make sure that a segment isn't
# too big as async tcp writes readiness of one connection might affect tail
# latencies of other connection. Not a problem with prempting runtimes (tokio)
max_segment_size = 10240
max_segment_count = 10
max_connections = 10001

Probably don't need all of those settings they are just the ones that I copy pasted from here

QuantumEntangledAndy commented 3 years ago

If you put neolink into debug mode with

export RUST_LOG=error,neolink=debug

Then you will see this for the debug messages

   Compiling neolink v0.3.0 (/Users/awk21/Projects/Software/neolink)
    Finished dev [unoptimized + debuginfo] target(s) in 16.53s
     Running `target/debug/neolink --config target/test.toml`
[2020-10-03T08:56:01Z INFO  neolink] Neolink v0.3.0-140-g0f83c7d debug
[2020-10-03T08:56:01Z DEBUG neolink] Create mqtt
[2020-10-03T08:56:01Z DEBUG neolink] Start mqtt server
[2020-10-03T08:56:01Z DEBUG neolink] Start polling mqtt
[2020-10-03T08:56:01Z DEBUG neolink] Send mqtt test
[2020-10-03T08:56:01Z INFO  neolink::mqtt::live] Starting MQTT Server
[2020-10-03T08:56:01Z DEBUG neolink] mqtt ready
[2020-10-03T08:56:01Z DEBUG neolink::mqtt::live] Incoming. Topic = /neolink, Payload = [b"start"]
[2020-10-03T08:56:01Z DEBUG neolink::gst] Permitting anonymous to access /Cammy02, /Cammy02/mainStream
[2020-10-03T08:56:01Z INFO  neolink] Cammy02: Connecting to camera at 192.168.1.101:9000
[2020-10-03T08:56:01Z INFO  neolink] Cammy02: Connected and logged in
[2020-10-03T08:56:01Z DEBUG neolink::bc_protocol::connection] Ignoring uninteresting message ID 78
[2020-10-03T08:56:01Z DEBUG neolink::bc_protocol::connection] Ignoring uninteresting message ID 79
[2020-10-03T08:56:01Z INFO  neolink] Cammy02: Camera time is already set: 2020-10-03 15:56:05 +7
[2020-10-03T08:56:01Z INFO  neolink] Cammy02: Starting video stream mainStream

Look out for these mqtt messages in the log to check its working

[2020-10-03T08:56:01Z DEBUG neolink] Create mqtt
[2020-10-03T08:56:01Z DEBUG neolink] Start mqtt server
[2020-10-03T08:56:01Z DEBUG neolink] Start polling mqtt
[2020-10-03T08:56:01Z DEBUG neolink] Send mqtt test
[2020-10-03T08:56:01Z INFO  neolink::mqtt::live] Starting MQTT Server
[2020-10-03T08:56:01Z DEBUG neolink] mqtt ready
[2020-10-03T08:56:01Z DEBUG neolink::mqtt::live] Incoming. Topic = /neolink, Payload = [b"start"]

You should be able to connect on port 8883 with your mqtt client and listen to messages

Topic = /neolink/Cammy02/status/motion, Payload = "on"
Topic = /neolink/Cammy02/status/motion, Payload = "off"

Where Cammy02 is your camera name

digiblur commented 3 years ago

Where do you set the mqtt server at? Mine is just a simple auth one as it stays local.

QuantumEntangledAndy commented 3 years ago

Neolink IS the MQTT server.

QuantumEntangledAndy commented 3 years ago

Hmmm would you rather run it as a client?

digiblur commented 3 years ago

Typically that is the case as I'd imagine most people doing home automation stuff with MQTT needs would already have a MQTT server running. I have one kicking with about 80 or 90 clients off of it already.

QuantumEntangledAndy commented 3 years ago

I see I see...

Oh well It was fun to make the server at least and learned lots doing it.

Give me a day or two and I'll swap it over a client. I'll call you again for a retest?

QuantumEntangledAndy commented 3 years ago

@digiblur I have a client based version up and running

Please download the latest version from same place as before.

Your config toml should look like this

[[cameras]]
name = "Cammy02"
username = "UserForCamera"
password = "passForCam"
address = "127.0.0.3:9000"
[cameras.mqtt]
server = "127.0.0.1"
port = 1883

[[cameras]]
name = "Cammy03"
username = "AnotherUserForCamera"
password = "passForAnotherCam"
address = "127.0.0.2:9000"
[cameras.mqtt]
server = "127.0.0.1"
port = 1883
credentials = ["User4MQTT", "Pass4MQTT"]

Each [[cameras]] must have a [cameras.mqtt] to post a message. Each camera runs its own client with the cilient id Neolink-CameraName.

The [cameras.mqtt] table supports the following options

[cameras.mqtt]
# The address of the mqtt server
server = "127.0.0.1" 

# The port of the mqtt server
port = 1883 

# Optional: User name and password for mqtt
credentials = ["User4MQTT", "Pass4MQTT"] 

# Optional: The servers encryption certificate
# Enabling this WILL turn on encryption
# You cannot use both ca and client_auth
ca = "/path/to/cert" 

# Optional: The clients encryption certificate and key
# Enabling this WILL turn on encryption
# You cannot use both ca and client_auth
client_auth = ["/Path/to/client/cert", "/Path/to/client/key"]
QuantumEntangledAndy commented 3 years ago

It also send the message

topic: /neolink/CameraName/start
message: start

When neolink starts the client

If there any other message you think we should send please let me know.

digiblur commented 3 years ago

Cool, I'll check it out. Is the start message configured as a LWT message to MQTT?

QuantumEntangledAndy commented 3 years ago

No idea what LWT is could you explain?

QuantumEntangledAndy commented 3 years ago

The start message is not a command just a report as a normal publish at the start.

QuantumEntangledAndy commented 3 years ago

P.s. it's getting late where I am so if you have issues I wont be able too help until morning. Hope it goes well :)

digiblur commented 3 years ago

No worries. Basically when the client logs in it posts a message as last will and testament with the retained flag set. If the MQTT server detects the client is gone for whatever reason it will post the unavailable payload of the LWT. It's great for knowing if a service, device, etc fell off the network or crashed etc. https://www.hivemq.com/blog/mqtt-essentials-part-9-last-will-and-testament/

QuantumEntangledAndy commented 3 years ago

Oh so that is what the retain flag does! Thanks :)

QuantumEntangledAndy commented 3 years ago

I think I can maintain two messages, one for neolink online/offline and one for camera connected/disconnected this should be a handy features thanks :)

QuantumEntangledAndy commented 3 years ago

Perhaps a single message with

digiblur commented 3 years ago

It does need to be specified as the LWT though so the broker can flip it to offline if the device/service no longer responds.

QuantumEntangledAndy commented 3 years ago

Yes I think I get it. Needs LWT and a retain

QuantumEntangledAndy commented 3 years ago

Is it working for you otherwise?

digiblur commented 3 years ago

Not sure. Is there a docker container of the change?

QuantumEntangledAndy commented 3 years ago

Umm do you need a docker?

QuantumEntangledAndy commented 3 years ago

Can't you just use the binary?

QuantumEntangledAndy commented 3 years ago

Can find the assets here

digiblur commented 3 years ago

Will see if I can swap it out.

QuantumEntangledAndy commented 3 years ago

Should be a matter of

QuantumEntangledAndy commented 3 years ago

You could try my docker

docker pull quantumentangledandy/neolink:mqtt

But I have not tested it

Be aware that setting the mqtt server to 127.0.0.1 or localhost will not work from inside a docker as it has its own seperate network. You have to use host.docker.internal instead

digiblur commented 3 years ago

I'll give it a shot today. Thanks!

digiblur commented 3 years ago

Cool I see the MQTT client logging in... drop the initial / of the topic as it creates a null initial entry on MQTT explorer and others. Motion events seem to send across just fine. Very cool! I'm not seeing the LWT topic working though. It should come across as a retain and LWT option. I brought the container down and it didn't post the offline message after getting a MQTT timeout.

I'd also like to see a motion event retain message as an option. For myself I'd combine with the use of LWT to determine if the device was online or not then use the retained state to see the last event even during a home automation system reboot.

Definitely cool to see MQTT messages off a motion event! Nice! Is there a way to send messages to Reolink cams? Like the Lumus to turn on the white LEDs?

QuantumEntangledAndy commented 3 years ago

Yes I haven't had time to setup the retain or LWT yet been busy with other things. But I agree that we should retain the on off status of the motion event since they do represent continuous states.

Thanks for the update on the leading / will change when I can.

With regards to control messages: We are currently reverse engineering more of the protocol in #80, I think we now know enough to control aspects such as the LED. I shall add this functionality into the MQTT for you to test if you'd like, but will have to wait until I have the time.