search

QuantumEntangledAndy / neolink

An RTSP bridge to Reolink IP cameras
GNU Affero General Public License v3.0
247 stars 39 forks source link

neolink does not detect cameras getting disconnected. #229

Open endreszabo opened 3 months ago

endreszabo commented 3 months ago

Describe the bug neolink does not detect cameras that get disconnected.

To Reproduce Steps to reproduce the behavior. Example:

  1. Create this configuration file: 
    
bind = "0.0.0.0"

[[cameras]] name = "kert" username = "admin" password = "foobar" address = "44.128.4.84:9000" stream = "mainStream" discovery = "local" print_format = "Human"

[mqtt] broker_addr = "127.0.0.1" port = 1883

[cameras.mqtt] enable_preview = false

2. Launch Neolink:
```console
$ docker run --rm --network host --name reolink --volume=$PWD/neolink.toml:/etc/neolink.toml quantumentangledandy/neolink neolink --config /etc/neolink.toml mqtt-rtsp
[2024-04-04T11:52:30Z INFO  neolink] Neolink df0642e0f8c5f14ac2979630b93a426e309d8361 release
[2024-04-04T11:52:30Z INFO  neolink::rtsp] Starting RTSP Server at 0.0.0.0:8554
[2024-04-04T11:52:30Z INFO  neolink::rtsp] kert: Rtsp Starting
[2024-04-04T11:52:30Z INFO  neolink::utils] kert: Connecting to camera at Address: 44.128.4.84:9000
[2024-04-04T11:52:30Z INFO  neolink_core::bc_protocol] kert: Trying TCP discovery
[2024-04-04T11:52:30Z INFO  neolink::mqtt] kert: MQTT Staring
[2024-04-04T11:52:30Z INFO  neolink_core::bc_protocol] kert: TCP Discovery success at 44.128.4.84:9000
[2024-04-04T11:52:30Z INFO  neolink::utils] kert: Logging in
[2024-04-04T11:52:30Z INFO  neolink::utils] kert: Connected and logged in
[2024-04-04T11:52:32Z INFO  neolink::common::camthread] kert: Camera time is already set: 2024-04-04 13:52:32.0 -01:00:00
[2024-04-04T11:52:34Z INFO  neolink::common::neocam] kert: Model RLC-843A
[2024-04-04T11:52:34Z INFO  neolink::common::neocam] kert: Firmware Version v3.1.0.2948_23112721
[2024-04-04T11:52:34Z INFO  neolink::rtsp::stream] kert: Available at /kert/main, /kert/Main, /kert/mainStream, /kert/MainStream, /kert/Mainstream, /kert/mainstream, /kert
  1. Disconnect the camera from the network. Wait a considerable amount of time. I waited 15+ mins.
  2. Observe that nothing happens on MQTT neolink topics.
  3. Reconnect the camera.

  4. Shortly, neolink will print:

    [2024-04-04T13:49:38Z WARN  neolink::common::camthread] kert: Connection Lost: IO Error: Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" }

Caused by:
    Connection reset by peer (os error 104)
[2024-04-04T13:49:38Z INFO  neolink::common::camthread] kert: Attempt reconnect in 50ms
[2024-04-04T13:49:38Z INFO  neolink::utils] kert: Connecting to camera at Address: 44.128.4.84:9000
[2024-04-04T13:49:38Z INFO  neolink_core::bc_protocol] kert: Trying TCP discovery
[2024-04-04T13:49:38Z INFO  neolink_core::bc_protocol] kert: TCP Discovery success at 44.128.4.84:9000
[2024-04-04T13:49:38Z INFO  neolink::utils] kert: Logging in
[2024-04-04T13:49:38Z INFO  neolink::utils] kert: Connected and logged in
[2024-04-04T13:49:40Z INFO  neolink::common::camthread] kert: Camera time is already set: 2024-04-04 15:49:40.0 -01:00:00
  5. neolink will send the following two MQTT messages at the same time: 
    neolink/kert/status disconnected
neolink/kert/status connected

Expected behavior I expected neolink to realize that a camera had been disconnected in real-time. Upon realizing this, neolink should have sent the neolink/kert/status with the value of disconnected right away. Maybe neolink should employ some protocol-level pinging/keepalive to check on the cameras whose sockets went silent.

Versions NVR software: n/a Neolink software: quantumentangledandy/neolink:latest (8daa1de9056e) Reolink camera model and firmware: Model RLC-843A, Firmware Version v3.1.0.2948_23112721 (latest as of now)

QuantumEntangledAndy commented 3 months ago

We actually do have pinging. We also drop the connection if there is no response for a certain timeframe. This is especially true with udp cameras since a camera must be dropped if no acknowledgement packet is sent within about 3s (camera will drop us too in this situation).

Perhaps it is not the ping but something causing the reconnect to lock up.

QuantumEntangledAndy commented 3 months ago

I notice your connecting over tcp instead of udp. There's no ack packets there and instead we rely on link packets to ping. I'll have a look into tcp keep alive ping