Cameras don't wake up from pause mode

[jaydkay]

Hi @QuantumEntangledAndy,

I have been experimenting a bit with neolink (0.5.0 / 0.5.1) and I've been successful to setup two cameras and view the stream using vlc.

Unfortunately, when I add the pause feature, the cameras never wake up to stream once neolink is started. It seems they are put into pause mode and never wake up again:

[2023-01-19T18:47:33Z INFO  neolink] Neolink 44675f25b41ece2acc2fb10f60f7395c59accc34 release
[2023-01-19T18:47:34Z WARN  neolink::rtsp::gst] Missing the gstreamer plugin `x264 (gst-plugins-ugly)` for `x264enc` element. Required to paused certain cameras
[2023-01-19T18:47:34Z WARN  neolink::rtsp::gst] Missing the gstreamer plugin `libav (gst-libav)` for `avdec_h264` element. Required to paused certain cameras
[2023-01-19T18:47:34Z WARN  neolink::rtsp::gst] Missing the gstreamer plugin `libav (gst-libav)` for `avdec_h265` element. Required to paused certain cameras
[2023-01-19T18:47:34Z INFO  neolink::rtsp] Starting RTSP Server at 0.0.0.0:8554
[2023-01-19T18:47:34Z INFO  neolink::rtsp::states] Garten: Connecting to camera at UID: 952700023NQHXKZI
[2023-01-19T18:47:34Z INFO  neolink::rtsp::states] Eingang: Connecting to camera at UID: 95270000ARITT6M5
[2023-01-19T18:47:36Z INFO  neolink::rtsp::states] Garten: Logging in
[2023-01-19T18:47:38Z INFO  neolink::rtsp::states] Garten: Successfully logged in
[2023-01-19T18:47:38Z INFO  neolink::rtsp::states] Eingang: Logging in
[2023-01-19T18:47:38Z INFO  neolink::rtsp::states] Garten: Camera time is already set: 2023-01-19 19:47:45 +1
[2023-01-19T18:47:39Z INFO  neolink::rtsp::states] Garten: Camera reports firmware version 1202_491_352_27
[2023-01-19T18:47:39Z INFO  neolink::rtsp::states] Garten: Starting stream
[2023-01-19T18:47:39Z INFO  neolink::rtsp::states::streaming] Garten: Starting video stream Main Stream (Clear)
[2023-01-19T18:47:39Z INFO  neolink::rtsp::states] Garten: Successfully started streaming
[2023-01-19T18:47:39Z WARN  neolink::rtsp] Not ready to pause
[2023-01-19T18:47:39Z INFO  neolink::rtsp::states] Eingang: Successfully logged in
[2023-01-19T18:47:39Z INFO  neolink::rtsp::states] Eingang: Camera time is already set: 2023-01-19 19:47:41 +1
[2023-01-19T18:47:39Z INFO  neolink::rtsp::states] Eingang: Camera reports firmware version 1221_246_237_22
[2023-01-19T18:47:39Z INFO  neolink::rtsp::states] Eingang: Starting stream
[2023-01-19T18:47:39Z INFO  neolink::rtsp::states::streaming] Eingang: Starting video stream Main Stream (Clear)
[2023-01-19T18:47:39Z INFO  neolink::rtsp::states] Eingang: Successfully started streaming
[2023-01-19T18:47:40Z INFO  neolink::rtsp] Pause on motion (start)
[2023-01-19T18:47:40Z INFO  neolink::rtsp::states] Garten: Pausing stream
[2023-01-19T18:47:40Z WARN  neolink::rtsp] Not ready to pause
[2023-01-19T18:47:40Z INFO  neolink::rtsp::states::paused] Garten: Starting paused stream Main Stream (Clear)
[2023-01-19T18:47:40Z INFO  neolink::rtsp::states] Garten: Successfully paused streaming
[2023-01-19T18:47:40Z INFO  neolink::rtsp] Pause on motion (start)
[2023-01-19T18:47:40Z INFO  neolink::rtsp::states] Eingang: Pausing stream
[2023-01-19T18:47:41Z INFO  neolink::rtsp::states::paused] Eingang: Starting paused stream Main Stream (Clear)
[2023-01-19T18:47:41Z INFO  neolink::rtsp::states] Eingang: Successfully paused streaming

As soon as I remove the [cameras.pause] sections in the config file, vlc connects again. Did I miss something?

Best regards, Jochen

[Checksum]

I see this as well. Have an Argus PT and the stream doesn’t resume if pause is enabled. Works fine without it though. I’m using the docker image with network=host.

Also let me say, thanks for the awesome work @QuantumEntangledAndy!

[QuantumEntangledAndy]

Ah thanks for the information. I thought I had fixed this one. I'll have to check it again. Does this happen with client/motion pause or both?

[Checksum]

Ah thanks for the information. I thought I had fixed this one. I'll have to check it again. Does this happen with client/motion pause or both?

Just tried with 0.5.1, and it didn't work with either client or motion pause. Once the stream is paused, connecting to it, or motion doesn't seem to restart it. Happy to grab debug logs if that'll help.

[jaydkay]

I am using both, so...

[cameras.pause]
on_motion = true
on_client = true
mode = "none"
timeout = 2.1

... and that doesn't seem to work, unfortunately.

[mferraci]

Hi,

I confirm getting same issue. When [cameras.pause] section is used in config, I am not able to connect to RTSP (but I can see CPU activity, so decode is happening). As soon as I remove the section, it works again.

I tried all combination on_motion /on_client without any success. It only works when section is fully removed.

[dkerr64]

Will weigh in here and report that I am seeing the same problem. Installed this version of Neolink into Ubuntu 22.04 and connected to Argus 3 Pro. Is working (sort of) if I delete the pause section from config.

[Hoppersusa]

Hi, sorry if this is in the wrong place.

I was trying Neolink ad5d09045cc21e7c7b00e8425e1bf1779ae16102 release neolink 0.5.2 to see if the pause issue was still there and it seems to only be able to login in the Argus Pro 2's I have when it tries to login in the Pro 3's I get: ERROR neolink::rtsp] Culvert: Fatal error: Culvert: Could not login to camera

Caused by:
    Camera responded with Service Unavaliable

Even with the Pro 2's after a minute I get this: ERROR neolink_core::bc_protocol::connection::bcconn] Deserialization error: Io(Custom { kind: Other, error: CameraTerminate })

What information can I provide that would be of help?

[dkerr64]

@Hoppersusa yup, see #29 I also have Argus 3 Pro and have the same problem, was just introduced yesterday. Andy is working on a fix.

[QuantumEntangledAndy]

I recently got an Argus2E and I can actually replicate this so that's quite useful.

What seems to happen is we request a stream the camera sends normal data for awhile then it seems to send junk data then forcibly disconnect.

I seem to be able mitigate this by 1. Only using 1 steam at a time and 2. Having a good connection such that we can acknowledge the udp packets quickly enough.

Udp packets use a buffer on the camera in order to resend missed packets to the client. I am wondering if that buffer fills up too fast when using two streams at once. But will keep testing.

[dkerr64]

Neolink should only pull one stream from the camera anyway. I was surprised that the default pulled all three streams, that seams very wasteful. I don't know if it has impact on battery life, but it certainly sends a lot more traffic over the WiFi than necessary. The camera may be located some distance from a WiFi access point and I have definitely observed difference in reliability based on WiFi strength. I just moved one of my access points to get a stronger connection.

[dkerr64]

When I configure pause mode I am not able to connect VLC to the rtsp steam. tail end of VLC's log...

live555 debug: we will now try HTTP tunneling mode
live555 debug: connection error -36
live555 error: Failed to connect with rtsp://192.168.17.174:8554/driveway/mainStream
main debug: no access_demux modules matched
main debug: creating access: rtsp://192.168.17.174:8554/driveway/mainStream
main debug: looking for access module matching "rtsp": 24 candidates
satip debug: try to open 'rtsp://192.168.17.174:8554/driveway/mainStream'
satip debug: connect to host '192.168.17.174'
main debug: net: connecting to 192.168.17.174 port 8554
main debug: connection succeeded (socket = 63)
main debug: net: opening 0.0.0.0 datagram port 9268
main debug: net: opening 0.0.0.0 datagram port 9269
satip error: Failed to setup RTSP session
main debug: net: connecting to 192.168.17.174 port 8554
main debug: connection succeeded (socket = 63)
access_realrtsp warning: Cseq mismatch, got 1, assumed 0
access_realrtsp debug: rtsp connected
access_realrtsp warning: only real/helix rtsp servers supported for now
main debug: no access modules matched
main debug: dead input
main debug: changing item without a request (current 0/1)
main debug: nothing to play

[QuantumEntangledAndy]

Neolink should only pull one stream from the camera anyway. I was surprised that the default pulled all three streams, that seams very wasteful

It's to do with how some people want to use the low quality stream for opencv analysis and then record with the HD one. Once pause mode is working well, this will help since we can then only pull the one in use

[Dinth]

Still a problem in 0.5.5

[QuantumEntangledAndy]

I'm ready to test my rework of the pause feature. It is currently here #57 with builds here https://github.com/QuantumEntangledAndy/neolink/actions/runs/4667707017 could anyone test it?

[Dinth]

Hi. Im using docker and im not that proficient to modify it with the new PR. But just wanted to add that i have noticed a weird behaviour - neolink periodically (but irregularly - sometimes after 8 hours, sometimes after 3 days) disconnects from the camera and cannot reconnect - from the end user perspective the behavior is same as with pause, and was not happening before with thirtythreefourty docker

[anthonws]

Will give it a go tomorrow and will report back! Thank you for such awesome new developments!

[QuantumEntangledAndy]

@Dinth You can test the docker at

docker pull quantumentangledandy/neolink:refractor-gts_rework

[Checksum]

I'm ready to test my rework of the pause feature. It is currently here #57 with builds here https://github.com/QuantumEntangledAndy/neolink/actions/runs/4667707017 could anyone test it?

I just gave the docker image a quick test - HUGE improvement in terms of the pause/resume streaming feature. Have an Argus PT and I can actually connect to the RTSP server using VLC now and view the stream, something that wasn't possible so far. Also, the CPU usage is looking so much better! Used to hammer my Pi 4 8GB before, but now well under normal range.

This is looking fantastic @QuantumEntangledAndy. Thanks for your hard work and tackling a hard technical problem 💯 . I'll do some more testing when I have some time.

[QuantumEntangledAndy]

@Checksum glad to hear all the hard work is helping. Getting the pause to play nicely was a tricky thing because the gstreamer and clients needs an initial stream of data to get things going.

Final solution was to keep a buffer of 100 frames to kinda replay for all new clients. This seemed to help with a lot of things as it meant that gstreamer had a large sample of data to work out the stream format right from the start reducing our connect times as well as reducing artifacts in clients because the replay stream always starts with an iframe (full video frame) rather than a pframe (update video frame)

Hoping to merge this and tackle some more issues. Next up will either be the mqtt or more pause enhancements.

[Dinth]

@Dinth You can test the docker at

docker pull quantumentangledandy/neolink:refractor-gts_rework

Im getting the following error using this docker:

[2023-04-12T08:24:49Z INFO  neolink] Neolink 50126a3833c796104f4b83842b5bca1f1d1d564a release

[2023-04-12T08:24:49Z WARN  neolink::rtsp] Without a server certificate, usernames and passwords will be exchanged in plaintext!

[2023-04-12T08:24:49Z WARN  neolink::rtsp::gst::factory] Constructing Factor Impl

[2023-04-12T08:24:49Z WARN  neolink::rtsp::gst::factory] Constructing Factor Impl

[2023-04-12T08:24:49Z INFO  neolink::rtsp] Starting RTSP Server at 0.0.0.0:8554

thread '<unnamed>' panicked at 'Could not build pipeline: Missing required gstreamer plugin `Unknown` for `textoverlay` element

Caused by:

    Failed to find element factory with name 'textoverlay' for creating element', src/rtsp/gst/factory.rs:157:51

note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

fatal runtime error: failed to initiate panic, error 5

Aborted (core dumped)

[QuantumEntangledAndy]

Your missing the textoverlay plugin. Could you maybe wait a minute before connecting the stream. I only use that plugin when you connect to fast to print a message that it's not ready yet.

[QuantumEntangledAndy]

Text overlay is meant to be in the base plugin set. So I'm not sure why it's missing in the docker.

[Dinth]

Just tried recreating docker with pull the image option checked in Portainer, but im still getting that error. Seems that neolink is connecting to the camera:

==Battery==
Charge: 100%,
Temperature: 32°C,
LowPower: false,
Adapter: solarPanel,
ChargeStatus: chargeComplete,
[2023-04-12T08:50:32Z INFO  neolink::rtsp::states::loggedin] baby_room: Camera time is already set: 2023-04-12 9:50:36 +0
[2023-04-12T08:50:32Z INFO  neolink::rtsp::states::loggedin] baby_room: Camera reports firmware version 1202_491_352_27
[2023-04-12T08:50:32Z INFO  neolink::rtsp::states::streaming] baby_room: Starting video stream Main Stream (Clear)

but im getting no stream from it

[QuantumEntangledAndy]

So I have added gstreamer1.0-plugins-base to the dockerimage that should mean textoverlay will be included but it will take about an hour for the updated docker image to be live

p.s. when you say pull the image option I am really not sure what you mean. I assume this is an option to update the image but does it update a tag or does it pull latest? If latest then thats not right one. Might be best to includ the hash so we can check the version that way. The next build should print this hash:

Neolink 7faddb3b50578b5a8be69bdbeaed87128b8c7fae release

But as I said its not ready yet

p.s. the current working one is 50126a3833c796104f4b83842b5bca1f1d1d564a but as youv'e pointed out the plugins-base is missing from the docker

[Dinth]

Sorry, i just removed from the config another camera which is currently offline, restarted neolink and got a stream now. But that was unexpected - before neolink never had problems with other streams when one of the cameras was offline

[QuantumEntangledAndy]

Not sure why that would be the case. They are all run on separate threads so shouldn't really care about each other

[QuantumEntangledAndy]

Updated docker with plugins-base should be live:

docker pull quantumentangledandy/neolink:refractor-gts_rework

[QuantumEntangledAndy]

Sorry, i just removed from the config another camera which is currently offline, restarted neolink and got a stream now. But that was unexpected - before neolink never had problems with other streams when one of the cameras was offline

Can't seem to replicate this. Tried to add offline and non existent camera to the config and was able stream a working camera no issues.

[anthonws]

This is a superb quality piece of code!

Camera: Argus 2E Firmware: 323_712_572_477

Config:

bind = "0.0.0.0"
bind_port = 8555

[[cameras]]
name = "tapas_balcony"
username = "admin"
password = "PASS"
uid = "000000000000000"
stream = "subStream"
  [cameras.pause]
  on_motion = true
  on_client = true 
  mode = "none"
  timeout = 2.1

Relevant observed indicators:

1. CPU usage is now sitting at 5/6%, instead of 100% (one core consumption) in my subpar mini pc celeron CPU (used for HA)
2. Stream opened every single time without any issues
3. Occasionally, the stream randomly stops (VLC), while streaming locally, without any indication in the logs other than the Client has paused. Enabled RUST trace env. variable but was not able to see any worthy piece of log that indicated the failure. Is there a specific trace env for the new gstreamer implementation?
4. In very rare occasions the stream presents some artifacts (green/purple blocks, and once it covered the entire screen and the stream would not recover)

Once more a big thank you for the phenomenal work that you have done here! Kudos!

Hth, anthonws.

[Checksum]

This issue seems to be sorted now. Both client and motion pause/resume works reliably over a period of 24 hours! Great work @QuantumEntangledAndy!

[QuantumEntangledAndy]

Ok I think I'll close it then. If anyone is still having trouble please ensure you can replicate on the latest release then ask me to reopen or make a new issue