search

QuantumEntangledAndy / neolink

An RTSP bridge to Reolink IP cameras
GNU Affero General Public License v3.0
322 stars 44 forks source link

Add support for floodlight control over MQTT #14

Closed kevin-david closed 1 year ago

kevin-david commented 1 year ago

This works... sometimes, but needs some more work/testing.

QuantumEntangledAndy commented 1 year ago

So what is up with the sometimes? Are there any telling errors?

QuantumEntangledAndy commented 1 year ago

Maybe start with a simpl cmd command rather than the mqtt interface so we can check if it the actual status light bit is working without the extra layer of difficulty that mqtt brings

QuantumEntangledAndy commented 1 year ago

How's it going I finished my rework of the pausable streams and I made some changes to mqtt not sure how that effects you so thought I'd check in here. My changes are currently on #17

kevin-david commented 1 year ago

Unfortunately I haven't been able to make time to look into this yet - I will try your suggestion next of just starting with a simple command, I think that's a good idea.

QuantumEntangledAndy commented 1 year ago

Alright I'll tackle mqtt next then and make sure everything is working there. Good luck with your hacking and let me know if you need some help. I recommend checking out src/pir/*, src/main.rs and src/cmdline.rs as those will take you though how to add a simple command

QuantumEntangledAndy commented 1 year ago

Sorry for the odd question but is your floodlight battery powered? I ask because I want to work out the camera messages for battery levels but I only own a non-battery camera.

kevin-david commented 1 year ago

Sorry for the odd question but is your floodlight battery powered? I ask because I want to work out the camera messages for battery levels but I only own a non-battery camera.

It is not - PoE powered! Here's the product page: https://reolink.com/product/reolink-floodlight/

QuantumEntangledAndy commented 1 year ago

Ah I see thanks anyway.

QuantumEntangledAndy commented 1 year ago

Did you ever find time to work on this? If not I can work something up.

kevin-david commented 1 year ago

Unfortunately I haven't - I was kind of waiting for an epiphany or Reolink to bail us out like I mentioned in the comment here: https://github.com/fwestenberg/reolink_dev/issues/599#issuecomment-1345356470

I do want (more) control of the floodlight/spotlight on the Duo Floodlight PoE camera I just got though - I am hoping the protocol is the same even if it's not exposed, specifically for the time the light is kept on after motion is detected, since according to Reolink that's "related to the duration of the alarm and the post-motion recording you set" right now and not able to be set on its own.

I am not seeing a lot of free time opening up in my schedule soon though :(

kevin-david commented 1 year ago

@QuantumEntangledAndy OK i am back at it here because I want to make my floodlight turn on another one :)

A few things that need to be changed...

Finally, the floodlight seems to drop off after some time, even with no commands being sent. Each time, it looks like:

[2023-01-25T04:30:25Z ERROR neolink_core::bc_protocol::connection::bcconn] Deserialization error: Deserialization error
[2023-01-25T04:30:27Z ERROR neolink_core::bc_protocol::connection::bcconn] caused by: I/O error
[2023-01-25T04:30:27Z ERROR neolink_core::bc_protocol::connection::bcconn] caused by: Read returned 0 bytes
[2023-01-25T04:30:27Z ERROR neolink::mqtt::event_cam] Motion thread aborted: Dropped connection

    Caused by:
        receiving on an empty and disconnected channel

What do you think about adding retry logic here, or making this a non-fatal error somehow? AFAIK from other languages, reading 0 bytes means "we're done" so that's a bit odd, maybe a quirk of this device?

QuantumEntangledAndy commented 1 year ago

This would be easier if I had some wireshark to look at.

There needs to be a subscriber somewhere to know where to the send the reply.

When you send a Bc packet you create a new message number. The camera will reply with the same message number. If your missing the subscriber or the subscriber is dropped then you get the message you described. I'll try and check your code and understand why you are not keeping your subscriber around long enough.

What does the official message flow look like. Do you request the status and then wait for a single reply or multiple replies?

QuantumEntangledAndy commented 1 year ago

I think you need to have a look at the offical client again in wireshark and find a message with ID MSG_ID_FLOODLIGHT_STATUS_LIST then filter the messages by msg_num for that message. It should show you the whole communication thread for that message from beginning to end.

QuantumEntangledAndy commented 1 year ago

What do you think about adding retry logic here, or making this a non-fatal error somehow? AFAIK from other languages, reading 0 bytes means "we're done" so that's a bit odd, maybe a quirk of this device?

If the connection is dropped we would have to start the whole connection from scratch and relogin. It is possible that there is a keep-alive message we could send peridocally. We have such a thing for the UDP connections but perhaps this device needs it even over TCP. Again seeing the wireshark would help

kevin-david commented 1 year ago

Packet captures attached: pcaps-reolink-floodlight.zip

I'll try and check your code and understand why you are not keeping your subscriber around long enough.

I'm not sure I'm setting it up properly at all, to be clear. I will keep digging in.

What does the official message flow look like. Do you request the status and then wait for a single reply or multiple replies?

As far as I can tell, there's a single request for multiple replies. At the end of the floodlight_comms_reolink_app_2 capture, you can see where I turn on and off the light with my phone (another client)

I think you need to have a look at the offical client again in wireshark and find a message with ID MSG_ID_FLOODLIGHT_STATUS_LIST then filter the messages by msg_num for that message. It should show you the whole communication thread for that message from beginning to end.

As far as I can tell (desktop client) this message is never sent client->camera, only camera->client. I can try to get a mobile app pcap again if you think that'd be useful.

It is possible that there is a keep-alive message we could send peridocally. We have such a thing for the UDP connections but perhaps this device needs it even over TCP. Again seeing the wireshark would help

I think you're right. From my read of the captures, the <LinkType> message (93) is being used as a keepalive, the TCP keep alives being sent by neolink aren't enough AFAICT.

QuantumEntangledAndy commented 1 year ago

Your messages seem atypical. Usually the message_handle field in the header links the message and the reply however in your dumps they are all 0 instead you have the channel_id doing what seems to be the usual msg_id's job.

Usually it works like this:

To match the reply with the message sent the msg_handle is used as a tag.

However your reolink app is doing it like this:

Your type 291's are also all on channel_id: 0 which means it is in reply to login. You'd need to subscribe to message ID 0 to capture them assuming that the floodgate does in fact send them when reolink connects.

It may also be that we are getting a disconnect from the floodgate because the message_handle is non-zero in neolink and the camera is complaining

QuantumEntangledAndy commented 1 year ago

Can you send two additional wireshark dumps:

  1. The camera using the Manual Floodlight Control message
  2. A full dump of neolink talking to the floodlight
QuantumEntangledAndy commented 1 year ago

I'm going to see if I can wrap up my rework into async code and UDP improvements then come and tackle this one with you. In the mean time you can try subscribing to msg_handle 0 and see what you can get

kevin-david commented 1 year ago

The camera using the Manual Floodlight Control message

Here you go, this is from the Android app, not the Mac app like I shared before: reolink_app_wireshark.zip. Manual control is message type 288, with 291 coming as a response after sending. <LinkType>/93 seems to be used as keepalives here as well

I accidentally managed to capture 2 versions because the IP of the device changed. When it couldn't contact it at 192.168.1.130, the app managed to find it at 192.168.1.131 and started communicating over UDP. Once I removed and readded the camera in the app, TCP worked. I removed the login messages in an attempt to not have to rotate my passwords a bunch of times, but if you need that let me know.

A full dump of neolink talking to the floodlight

This was in the original ZIP - it's floodlight_comms_neolink_mqtt_2.pcapng. Happy to get another one with a different command if you want.

kevin-david commented 1 year ago

I was able to get this partially working again! Publishing an MQTT message to the broker (i.e. neolink/backyard floodlight/control/floodlight set to raw on or off works until the camera disconnects due to the missing <LinkType> keepalives.

So the big things missing are:

QuantumEntangledAndy commented 1 year ago

How did it go. I'd like to pick this up but would need some help since you have the actual device

QuantumEntangledAndy commented 1 year ago

I've merged master in so that it should be using the new async methods now

QuantumEntangledAndy commented 1 year ago

We have a method for regsitering a callback for when the camera sends a message so we can use that for firing off the right mqtt messages

QuantumEntangledAndy commented 1 year ago

I added the callback so we can listen for the floodlight events on 291

I also added the LinkType pings

I really hope that works for you

QuantumEntangledAndy commented 1 year ago

Floodlight on/off are reported to the status/floodlight mqtt topic

kevin-david commented 1 year ago

Awesome! Pulled it and tried it out. Two discoveries so far:

thread 'tokio-runtime-worker' panicked at 'Cannot block the current thread from within a runtime. This happens because a function attempted to block the current thread while the thread is being used to drive asynchronous tasks.', 

/Users/kevin/dev/neolink/crates/core/src/bc_protocol/floodlight_status.rs:31:32

I've tried a couple different things so far to fix it but I am way out of my depth with Rust...

QuantumEntangledAndy commented 1 year ago

So it was a little difficult but I swapped it to using async callbacks so that we can call send().await inside the callback. Please have another try

kevin-david commented 1 year ago

That definitely looks tricky! T: 'static + Send + Sync + for<'a> Fn(&'a Bc) -> BoxFuture<'a, Option<Bc>>,1 - whew. Explains why I couldn't figure it out 😄

I made one last change while debugging - I set up the XML deserialization wrong, but it is working now.

Two future potential changes - curious if you're OK with either of these and have a preferred approach? I'm happy to take another look at either of these for a future PR:

QuantumEntangledAndy commented 1 year ago

I'm quite curious about mqtt discovery. That seems worth pursuing.

QuantumEntangledAndy commented 1 year ago

Seems ok. LGTM

kevin-david commented 1 year ago

Awesome, thanks for all the help on this! I'll take a look at MQTT discovery next and send a PR once I've got something going. Trying to use this as an excuse to better learn Rust anyway 😄

QuantumEntangledAndy commented 1 year ago

If you need any rust pointers let me know. Neolink was my first rust project too

QuantumEntangledAndy commented 1 year ago

Also I meant to ask. When you login with debug log enabled there should be a message about the abilities the camera supports. Can you send that over? It would be nice to compare that to a camera. Could maybe auto detect flood light features and when camera is not possible.

kevin-david commented 1 year ago

@QuantumEntangledAndy Sure! This is for two different devices, the floodlight and the Duo PoE camera with floodlights as well. I think the second one below is the floodlight without a camera given the response.

<?xml version="1.0" encoding="utf-8"?>
<AbilityInfo>
    <userName>admin</userName>
    <system>
        <subModule>
            <abilityValue>general_rw, norm_rw, version_ro, uid_ro, autoReboot_rw, restore_rw, reboot_rw, shutdown_rw, dst_rw, log_ro, output_rw, performance_ro, upgrade_rw, export_rw, import_rw, bootPwd_rw</abilityValue>
        </subModule>
    </system>
    <network>
        <subModule>
            <abilityValue>port_rw, dns_rw, email_rw, ipFilter_rw, localLink_rw, pppoe_rw, upnp_rw, ntp_rw, netStatus_rw, ptop_rw</abilityValue>
        </subModule>
    </network>
    <alarm>
        <subModule>
            <abilityValue>hddFull_rw, hddError_rw, disconnect_rw, ipConflict_rw, rfAlarm_rw</abilityValue>
        </subModule>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>motion_rw, videoLost_rw, hide_rw</abilityValue>
        </subModule>
    </alarm>
    <image>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>ispBasic_rw</abilityValue>
        </subModule>
    </image>
    <video>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>osdName_rw, osdTime_rw, shelter_rw</abilityValue>
        </subModule>
    </video>
    <replay>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>replay_rw, seek_rw</abilityValue>
        </subModule>
    </replay>
    <PTZ>
        <subModule>
            <abilityValue>control_rw, preset_rw, cruise_rw, track_rw, decoder_rw, ptzInfo_ro</abilityValue>
        </subModule>
    </PTZ>
    <streaming>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>preview_rw, compress_rw, snap_rw, rtsp_rw, streamTable_ro</abilityValue>
        </subModule>
    </streaming>
</AbilityInfo>
<AbilityInfo>
    <userName>admin</userName>
    <system>
        <subModule>
            <abilityValue>general_rw, norm_rw, version_ro, uid_ro, autoReboot_rw, restore_rw, reboot_rw, shutdown_rw, dst_rw, log_ro, performance_ro, upgrade_rw, export_rw, import_rw, bootPwd_rw</abilityValue>
        </subModule>
    </system>
    <network>
        <subModule>
            <abilityValue>port_rw, dns_rw, email_rw, ftp_rw, ftpSchedule_rw, ipFilter_rw, localLink_rw, pppoe_rw, upnp_rw, ntp_rw, netStatus_rw, ptop_rw, autontp_rw</abilityValue>
        </subModule>
    </network>
    <alarm>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>motion_rw</abilityValue>
        </subModule>
    </alarm>
    <image>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>ispBasic_rw, ispAdvance_rw, ledState_rw</abilityValue>
        </subModule>
    </image>
    <video>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>osdName_rw, osdTime_rw, shelter_rw</abilityValue>
        </subModule>
    </video>
    <security>
        <subModule>
            <abilityValue>user_rw, userOnline_rw, bootPwd_rw</abilityValue>
        </subModule>
    </security>
    <PTZ>
        <subModule>
            <abilityValue>control_rw, preset_rw, cruise_rw, track_rw, decoder_rw, ptzInfo_ro</abilityValue>
        </subModule>
    </PTZ>
    <streaming>
        <subModule>
            <channelId>0</channelId>
            <abilityValue>preview_rw, compress_rw, snap_rw, rtsp_rw, streamTable_ro</abilityValue>
        </subModule>
    </streaming>
</AbilityInfo>
kevin-david commented 1 year ago

I'm quite curious about mqtt discovery. That seems worth pursuing.

Took a first attempt in https://github.com/QuantumEntangledAndy/neolink/pull/95!