ONVIF to NVR losing connection

[myromeo]

I have connected two yi home 1080p cameras with your firmware, very nice work by the way, but I have a strange issue.

I've got both cameras connecting to my NVR (Hikvision) via ONVIF and have set a username and password via the webif. They initially connect and work great but after a whilst they drop the connection. NVR reports the cameras as online and IP addresses have not changed. I can access both cameras via yi home app... I need to remove them from the NVR and re-add them to re-establish connection.

Any ideas? I saw a discussion regarding ONVIF security. Would it be better to remove user/pass and connect without authentication? Its an internal NVR so no security risk.

[roleoroleo]

Try without security.

[myromeo]

Removed username and password on webif so both blank. NVR will not accept blank username but webif will. Strange, if webif user/pass are blank then NVR can have any user/pass and it still connects. Maybe authentication is not working correctly?

Cameras still disconnecting although now reboot reconnects to NVR.

[roleoroleo]

Removed username and password on webif so both blank. NVR will not accept blank username but webif will. Strange, if webif user/pass are blank then NVR can have any user/pass and it still connects. Maybe authentication is not working correctly?

Currently, I don't use authentication but I think is working correctly.

Cameras still disconnecting although now reboot reconnects to NVR.

When cam disconnects, are you able to connect it using vlc or onvif dm?

[myromeo]

Removed authentication. Was still disconnecting.

Checked all other settings and found record on motion disabled (always record). Changed this back to on motion only and not enabled in motion settings so never record.

Connection now stable.

For info, when the camera disconnects I was still able to connect via webif and yi home apps so still connected, just lost onvif.

[roleoroleo]

Probably is a cpu load issue. Just a question: when the cam disconnects does rtsp stream work?

[myromeo]

I’d have to recreate the fault as I don’t know about RTSP, I’m not using that so never checked. Reluctant to recreate as its working now :)

[roleoroleo]

No problem.

[myromeo]

So they went off again after 12 hours or so. Something amiss. Still accessible via webif and yi home app. Simple restart fixes onvif. I wonder if onvif service would benefit from a periodic restart? Is that possible?

Firmware Version | 0.2.6 Base Version | 4.5.0.0C_201910080934 Model Suffix | y203c Serial Number | 6FUS*** Local Time | Sat Feb 8 17:34:14 UTC 2020 Uptime | 19h 50m 36s Load Average | 1.55 1.80 1.79 Free/Total Memory | 18612/60824 KB

[roleoroleo]

Maybe a memory leak. I will check it. Probably your onvif client call the cam continuously.

[myromeo]

Thank you.

My client is Hikvision NVR, I’m not sure it calls it all the time when not being viewed? I’m not recording or monitoring but perhaps it does. Memory leak would make sense.

When it works it works great, be amazing to resolve this little issue.

[roleoroleo]

I checked it and there aren't memory leaks. The mystery deepens!

[skylarhays]

This happens with ONVIF Device manager for windows as well... it works for a long while then it just drops. I can open another one of my cams and switch back and the stream will restart. It seems like something is wonky with ONVIF.

[myromeo]

Mines been quite stable the last few days, it seems random durations. I’ve tried turning off other services like RTSP, ftp etc to lighten the load but not found anything that makes a difference. Running default settings with no security now. Simple reboot resolves as if onvif service is stalling

[myromeo]

Little update. I’ve been monitoring this issue and have noticed that when the connection is lost I also lose access to webif for a few minutes. The webif restarts automatically ok but onvif does not without a reboot. Camera remains accessible via yi app.

[roleoroleo]

This is very strange because there is no restart procedure for webif. I'm beginning to believe that it is not a onvif problem but a rtsp problem.

[myromeo]

Is there anything I can test to help?

[roleoroleo]

If you want, I can give you a script to test. Add this file to /home/yi-hack/script/ directory and rename it wd_rtsp.sh with permission 700.

#!/bin/sh

CONF_FILE="etc/system.conf"

YI_HACK_PREFIX="/home/yi-hack"

#LOG_FILE="/tmp/sd/wd_rtsp.log"
LOG_FILE="/dev/null"
COUNTER=0
COUNTER_LIMIT=5
INTERVAL=10

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/lib:/home/yi-hack/lib:/tmp/sd/yi-hack/lib
export PATH=$PATH:/home/base/tools:/home/yi-hack/bin:/home/yi-hack/sbin:/tmp/sd/yi-hack/bin:/tmp/sd/yi-hack/sbin

get_config()
{
    key=$1
    grep -w $1 $YI_HACK_PREFIX/$CONF_FILE | cut -d "=" -f2
}

restart_rtsp()
{
        if [[ $(get_config RTSP) == "yes" ]] ; then
                if [[ $(get_config RTSP_HIGH) == "yes" ]] ; then
                        h264grabber -r high | RRTSP_RES=0 RRTSP_PORT=$RTSP_PORT RRTSP_USER=$USERNAME RRTSP_PWD=$PASSWORD rRTSPServer &
                else
                        h264grabber -r low | RRTSP_RES=1 RRTSP_PORT=$RTSP_PORT RRTSP_USER=$USERNAME RRTSP_PWD=$PASSWORD rRTSPServer &
                fi
        fi
}

check_rtsp()
{
#       echo "$(date +'%Y-%m-%d %H:%M:%S') - Checking RTSP process..." >> $LOG_FILE
        SOCKET=`netstat -an 2>&1 | grep ":554 " | grep ESTABLISHED | grep -c ^`
        CPU_1=`top -b -n 1 | grep h264grabber | grep -v grep | awk '{print $8}'`
        CPU_2=`top -b -n 1 | grep rRTSPServer | grep -v grep | awk '{print $8}'`

        if [ $SOCKET -eq 0 ]; then
                if [ "$CPU_1" == "" ] || [ "$CPU_2" == "" ]; then
                        echo "$(date +'%Y-%m-%d %H:%M:%S') - No running process, restarting..." >> $LOG_FILE
                        killall -q rRTSPServer
                        killall -q h264grabber
                        sleep 1
                        restart_rtsp
                fi
                COUNTER=0
        fi
        if [ $SOCKET -gt 0 ]; then
                if [ "$CPU_1" == "0.0" ] && [ "$CPU_2" == "0.0" ]; then
                        COUNTER=$((COUNTER+1))
                        echo "$(date +'%Y-%m-%d %H:%M:%S') - Detected possible locked process ($COUNTER)" >> $LOG_FILE
                        if [ $COUNTER -ge $COUNTER_LIMIT ]; then
                                echo "$(date +'%Y-%m-%d %H:%M:%S') - Restarting process" >> $LOG_FILE
                                killall -q rRTSPServer
                                killall -q h264grabber
                                sleep 1
                                restart_rtsp
                                COUNTER=0
                        fi
                else
                        COUNTER=0
                fi
        fi
}

if [[ x$(get_config USERNAME) != "x" ]] ; then
    USERNAME=$(get_config USERNAME)
    PASSWORD=$(get_config PASSWORD)
fi

case $(get_config RTSP_PORT) in
    ''|*[!0-9]*) RTSP_PORT=554 ;;
    *) RTSP_PORT=$(get_config RTSP_PORT) ;;
esac

echo "$(date +'%Y-%m-%d %H:%M:%S') - Starting RTSP watchdog..." >> $LOG_FILE

while true
do
        check_rtsp
        if [ $COUNTER -eq 0 ]; then
                sleep $INTERVAL
        else
                sleep 1
        fi
done

Add the line $YI_HACK_PREFIX/script/wd_rtsp.sh & to /home/yi-hack/script/system.sh:

...
if [[ $(get_config RTSP) == "yes" ]] ; then
    if [[ $(get_config RTSP_HIGH) == "yes" ]] ; then
        h264grabber -r high | RRTSP_RES=0 RRTSP_PORT=$RTSP_PORT RRTSP_USER=$USERNAME RRTSP_PWD=$PASSWORD rRTSPServer &
        ONVIF_PROFILE_0="--name Profile_0 --width 1920 --height 1080 --url rtsp://%s$D_RTSP_PORT/ch0_0.h264 --snapurl http://%s$D_HTTPD_PORT/cgi-bin/snapshot.sh?res=high --type H264"
    else
        h264grabber -r low | RRTSP_RES=1 RRTSP_PORT=$RTSP_PORT RRTSP_USER=$USERNAME RRTSP_PWD=$PASSWORD rRTSPServer &
        ONVIF_PROFILE_1="--name Profile_1 --width 640 --height 360 --url rtsp://%s$D_RTSP_PORT/ch0_1.h264 --snapurl http://%s$D_HTTPD_PORT/cgi-bin/snapshot.sh?res=low --type H264"
    fi
    $YI_HACK_PREFIX/script/wd_rtsp.sh &
fi
...

And restart the cam.

[myromeo]

So that didn’t work, camera is now not accessible via web, onvif or yi app. Boots up and blue light goes steady as if all is good but I can not use it. Help!

[roleoroleo]

Probably a syntax error. I will send a new beta firmware.

[myromeo]

thank you

[roleoroleo]

Which model? y203c, h201c or y25?

[myromeo]

y203c

[roleoroleo]

Here it is: y203c_0.2.7.tar.gz

[myromeo]

Thanks. 0.2.7 installed and up and running. I will let you know how it performs. I’ve left one camera on 0.2.6 as comparison.

[myromeo]

0.2.7 stable after 36 hours and counting. 0.2.6 dropped once at about 18 hours. I’ll continue monitoring and keep you informed.

[roleoroleo]

Ok, in the meantime I have further improved the watchdog algorithm.

[roleoroleo]

Please try release 0.2.7

[myromeo]

Seems stable on 0.2.7 so far. Will close this now. Thank you!

New issue, one of two cameras losing connection to iVMS, error 8200. This seems a common issue with iVMS so I'll explore known solutions before raising this as an issue! I've disabled platform access on both cameras as that seems a common issue. Lets see how it goes.