MQTT topics stops publishing after sometime

[MaximumSU]

I have two cameras connected to hass via mosquitto mqtt broker.

One:

Camera Model	yi_home_1080p
Firmware Version	0.2.0
Base Version	2.1.0.0E_201809191630

Second:

Camera Model	yi_dome
Firmware Version	0.2.0
Base Version	1.9.1.0J_201809191135

The First one works without issues, but second stops publishing any motion topics after sometime (approximitely after one day of work). After reboot it works for the second day, and then stop again. RTSP and cloud works well, debug_alarm.txt log is also updated as it.

156

[tinysnake]

yes, according to @TheCrypt0 's mqtt code, he must written the mqtt code in a hurry. what if arduino pubsubclient works

[MaximumSU]

Looks like it happened if many events occurred. First cam have very few events, but second cam in place, where it constantly start/stop recording. But, if events is few for the second cam, it publishing topics without problems. Now i have workaround - i restart the mqttv4 in camera from mqtt broker by ssh at every night.

[gooman-uk]

It's quite possibly related to this: https://github.com/TheCrypt0/yi-hack-v4/issues/65

Short version is that as @tinysnake alluded to, the existing code doesn't do reconnects if the connection drops.

Someone who knows how to wrestle with Github should try the code I showed in that post.

[MaximumSU]

Short version is that as @tinysnake alluded to, the existing code doesn't do reconnects if the connection drops.

I d't think so. Now it happened again and i have the log. If all good, it look's like:

CALLBACK MOTION START Client cam_dinner_1 sending PUBLISH (d0, q1, r1, m19, 'yicam/motion_cam_dinner_1', ... (12 bytes)) Client cam_dinner_1 received PUBACK (Mid: 19) Client cam_dinner_1 sending PINGREQ Client cam_dinner_1 received PINGRESP Client cam_dinner_1 sending PINGREQ Client cam_dinner_1 received PINGRESP Client cam_dinner_1 sending PINGREQ Client cam_dinner_1 received PINGRESP CALLBACK MOTION STOP Client cam_dinner_1 sending PUBLISH (d0, q1, r1, m20, 'yicam/motion_cam_dinner_1', ... (11 bytes)) Client cam_dinner_1 received PUBACK (Mid: 20)

but after that:

CALLBACK MOTION START Client cam_dinner_1 sending PUBLISH (d0, q1, r1, m21, 'yicam/motion_cam_dinner_1', ... (12 bytes)) Client cam_dinner_1 received PUBACK (Mid: 21) Client cam_dinner_1 sending PINGREQ Client cam_dinner_1 received PINGRESP Client cam_dinner_1 sending PINGREQ Client cam_dinner_1 received PINGRESP CALLBACK MOTION START Client cam_dinner_1 sending PUBLISH (d0, q1, r1, m22, 'yicam/motion_cam_dinner_1', ... (12 bytes)) Client cam_dinner_1 received PUBACK (Mid: 22) Client cam_dinner_1 sending PINGREQ CALLBACK MOTION STOP CALLBACK MOTION START CALLBACK MOTION START CALLBACK MOTION STOP CALLBACK MOTION START CALLBACK MOTION STOP CALLBACK MOTION START CALLBACK MOTION START CALLBACK MOTION STOP

as you see, after publishing the correct "MOTION STOP" message there are two "MOTION START" messages and then only the "CALLBACK MOTION STOP" without realy publishing and after that only "CALLBACKS" wthiout any publishing. Also there are no any disconnects or so... But, i d't know, may be the such message not provided by the code....

[bmangodt]

Same issue (I think) Camera won't publish any messages whatsoever... Not even a reboot fixes the issue on my cameras. The amount of events or connection drops don't matter. I was thinking it was a problem in my Mosquito broker on Hass.io but all the other MQTT connected devices work just fine.

[tnTano]

same problem... for my camera the reboot do not works... they are the only device do not publish topic...

did someone have any idea?

thanks

[roleoroleo]

I created a PR in the mqtt project. On my system it has been running for 10 days.

[tnTano]

I created a PR in the mqtt project. On my system it has been running for 10 days.

i can't find the project can you link it? just for checking the roadmap and current status...

thanks

[roleoroleo]

https://github.com/TheCrypt0/mqttv4

[tnTano]

https://github.com/TheCrypt0/mqttv4

thanks!

[stcbus]

I'm seeing this as well.

[tnTano]

no one have some news??

[tnTano]

@roleoroleo with you PR the problem was resolved?

if yes there is a way to compile you change in local and use it?

[roleoroleo]

I think the problem is resolved.

Follow the wki to install SDK environment: https://github.com/TheCrypt0/yi-hack-v4/wiki/Project-Setup-Hints

After the git clone command, apply manually the patch that I posted in the PR. And try to compile:

./scripts/compile.sh

Skip the init and the pack commands: you don't have to build an image but only to compile the app.

[tnTano]

hi @roleoroleo , one thing, i need original firmware? because i try to download them from yi website but i think they doesn't works...

[roleoroleo]

No, you don't need the original firmware if you don't want to build the full image. After the compile.sh command, get the mqttv4 binary from _install directory and copy it to the cam.

[tnTano]

sorry @roleoroleo but i cannot install all the sdk, after following all the installation guide for the HI SDK I found an error that is not tracked:

See `config.log' for more details.
make[2]: Entering directory '/root/Hi3518E_SDK_V1.0.4.0/osdrv/tools/pc/lzma_tool/lzma-4.32.7'
make[2]: *** No targets specified and no makefile found.  Stop.
make[2]: Leaving directory '/root/Hi3518E_SDK_V1.0.4.0/osdrv/tools/pc/lzma_tool/lzma-4.32.7'
~/Hi3518E_SDK_V1.0.4.0/osdrv/tools/pc/lzma_tool
cp: cannot stat '/root/Hi3518E_SDK_V1.0.4.0/osdrv/tools/pc/lzma_tool/lzma-4.32.7/src/lzma/lzma': No such file or directory
Makefile:6: recipe for target 'all' failed
make[1]: *** [all] Error 1
make[1]: Leaving directory '/root/Hi3518E_SDK_V1.0.4.0/osdrv/tools/pc/lzma_tool'
Makefile:204: recipe for target 'hipctools' failed
make: *** [hipctools] Error 2

i try to run the ./scripts/compile.sh

 ------------------------------------------------------------------------
  YI-HACK-V4 - SRC COMPILER
 ------------------------------------------------------------------------

 MOD_DIR:        /root/yi-hack-v4/src/busybox
 MOD_NAME:       busybox
 MOD_INIT:       init.busybox
 MOD_COMPILE:    compile.busybox
 MOD_INSTALL:    install.busybox

 Initializing busybox...

 HEAD is now at e70c38f [Edit] Added #supporters info to the readme
 Compiling busybox...

 make: *** No targets specified and no makefile found.  Stop.

[roleoroleo]

You don't need to build the SDK. Get the crosstool environment (arm-hisiv300-linux.tar.bz2) and extract it to your filesystem. However try this: mqttv4.gz

[tnTano]

i'll try it

thanks....

[kirilldem]

You don't need to build the SDK. Get the crosstool environment (arm-hisiv300-linux.tar.bz2) and extract it to your filesystem. However try this: mqttv4.gz

Is there a guide somewhere with more detailed instructions?

[photonforge]

https://github.com/TheCrypt0/mqttv4

will replacing the mqttv4 on the sd card with this file via ftp work?

[tnTano]

i replace the mqtt file but nothing change... i'll reconfig all the mqtt but i have bad feeling with this...

[roleoroleo]

You need to unzip the file and copy it to the cam using ftp. If you want, rename the original mqttv4 binary, you will be able to restore it if necessary. Then, connect with putty and verify the permission of the mqttv4 file. It must be executable (i.e. rwxrwxrwx).

@tnTano What's the problem? The program crashes?

[tnTano]

i replace the file, i set the permission, i reconfigured all the settings for mqtt this is the conf on binary_sensor ha:

- platform: mqtt
  state_topic: "tvcc-corridoio/motion"
  name: "motion_tvcc-corridoio"
  payload_on: "on"
  payload_off: "off"
  device_class: motion

but nothing happen... not working...

[MaximumSU]

i replace the file, i set the permission, i reconfigured all the settings for mqtt

Are you restarted the camera after all? I also stoped the daemon before the copy new one by: killall mqttv4 It works, the log:

Starting mqttv4 v0.0.3
Trying to connect... Client cam_dinner_1 sending CONNECT
Client cam_dinner_1 received CONNACK (0)
. connected!
Client cam_dinner_1 sending PINGREQ
Client cam_dinner_1 received PINGRESP
.......................
CALLBACK MOTION START
Client cam_dinner_1 sending PUBLISH (d0, q1, r1, m8, 'yicam/motion_cam_dinner_1', ... (12 bytes))
Trying to connect... Unable to connect (Connection timed out).
Client cam_dinner_1 sending CONNECT
Client cam_dinner_1 received CONNACK (0)
Client cam_dinner_1 sending PUBLISH (d1, q1, r1, m7, 'yicam/motion_cam_dinner_1', ... (11 bytes))
Client cam_dinner_1 sending PUBLISH (d1, q1, r1, m8, 'yicam/motion_cam_dinner_1', ... (12 bytes))
. connected!
Client cam_dinner_1 received PUBACK (Mid: 7)
Client cam_dinner_1 received PUBACK (Mid: 8)
Client cam_dinner_1 sending PINGREQ
Client cam_dinner_1 received PINGRESP
.....................

[tnTano]

i replace the file stopping process... i think the log was under /tmp/log.txt but i cant find something with MOTION or something else... @MaximumSU there are other logs to view??

[MaximumSU]

i replace the file stopping process but i cant find log... @MaximumSU where you saw that?

start the mqtt something like this: mqttv4 > /tmp/mqttv4_log 2>&1 & but be carefully - the log will grou up without properly cleaning it by you, system does not care about it.

[tnTano]

Trying to connect... Client tvcc-corridoio sending CONNECT
Client tvcc-corridoio received CONNACK (5)
Connection Refused: not authorised.
......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... timeout!

after some times it went to timeoute... i don't understand... the mqtt config are correct...

[MaximumSU]

Connection Refused: not authorised.

it is here. Check the Login/Password settings on Server and Client, check the Server's mqtt log. It may be also the firewall (if any) issue.

[tnTano]

ok... there are some limitation of character in password ?? My mqtt pass start with $... with other thinks works without problem but i know there are some cases that use the $ for the first character generate big problems...

[bmangodt]

When I try to start the mqttv4 service, all I get is: "Illegal instruction". Anyone have any idea? Permissions are set and I have rebooted the camera several times

[MaximumSU]

ok... there are some limitation of character in password ??

Yes, it is discribed here: under the section "Known issues" But there is workaround: you can manually edit the file /home/yi-hack-v4/etc/mqttv4.conf with the correct value.

[tnTano]

i found and resolve my problem.... the password field (or the variable) do not handle special characters but transform them into% number so if you have special characters in the password mqtt enter it directly in the file /home/yi-hack-v4/etc/mqtt4.conf

now it remains to understand how to save the recording in a remote folder with HA or with nodered

thanks to all for the help

[Minims]

@MaximumSU many thanks for this. I hope it will be released a day :-)

[Pako9004]

In my case i have a outdoor version of yi camera, and when i use the mqttv4 modified file, all I get is: "Illegal instruction". Anyone have any idea?

Permissions are set and I have rebooted the camera several times

[Minims]

Where do you se this log, it seems working on my outdoor camera.

[Pako9004]

When i restart the camera (in the log) or if i execute mqttv4 from command line.

[Minims]

what is the md5sum of your mqttv4 file ?

/home/yi-hack-v4 # md5sum mqttv4* cbb97f38101a2938134ca15b7c8e4851 mqttv4 3ecbaa750ec8640f15f8aad63afa6e95 mqttv4.orig cbb97f38101a2938134ca15b7c8e4851 mqttv4.patched

[Pako9004]

I think is the same:

cbb97f38101a2938134ca15b7c8e4851 mqttv4

[Pako9004]

I have repeated the process completely and now it works! I don't know what the mistake was ...

Thank you @Minims

[bmangodt]

I still have the same issue as @Pako9004 @Minims. when I do the md5sum command it says: 'No such file or directory'. I have to navigate to the /bin folder for it to find it and then it shows: '1f09bc8ce035c169bd62c832e95109aa mqttv4'. Any ideas? This is like my 10th time trying to change the file and every time I have the Illegal instruction error...

[Pako9004]

Hello @bmangodt

I think that i found the problem. If i use filezilla to send the file to ftp the md5sum changes. Now i use gFTP in Ubuntu. Maybe you can prove another ftp client and check again.

[bmangodt]

Hello @bmangodt

I think that i found the problem. If i use filezilla to send the file to ftp the md5sum changes. Now i use gFTP in Ubuntu. Maybe you can prove another ftp client and check again.

Hello @Pako9004! That worked like a charm my friend! I used Cyberduck instead of FileZilla and the checksum is valid now! Thank you!

[kirilldem]

Looks like it happened if many events occurred. First cam have very few events, but second cam in place, where it constantly start/stop recording. But, if events is few for the second cam, it publishing topics without problems. Now i have workaround - i restart the mqttv4 in camera from mqtt broker by ssh at every night.

Could you share the script here? I would like to restart the cam from my raspberry pi nightly.

Many thanks!

[MaximumSU]

@kirilldem

Could you share the script here? I would like to restart the cam from my raspberry pi nightly.

Sure. The script:

#!/bin/bash

password=your_root_password
cam_ip=your_cam_ip

sshpass -p $password ssh root@$cam_ip "export PATH=/usr/bin; killall rtspv4 >> /tmp/rtspv4_log 2>&1"
sshpass -p $password ssh root@$cam_ip "export PATH=/usr/bin; killall viewd >> /tmp/viewd_log 2>&1"
sshpass -p $password ssh root@$cam_ip "export PATH=/usr/bin; killall mqttv4 >> /tmp/mqttv4_log 2>&1"

sshpass -p $password ssh root@$cam_ip "export PATH=/usr/bin:/usr/sbin:/bin:/sbin:/home/base/tools:/home/app/localbin:/home/base:/home/yi-hack-v4/bin:/home/yi-hack-v4/sbin:/tmp/sd/yi-hack-v4/bin:/tmp/sd/yi-hack-v4/sbin; export LD_LIBRARY_PATH=/usr/local/lib:/usr/lib:/home/lib:/home/hisiko/hisilib:/home/app/locallib:/home/yi-hack-v4/lib:/tmp/sd/yi-hack-v4/lib; echo `date` > /tmp/viewd_log 2>&1; viewd >> /tmp/viewd_log 2>&1 &"
sshpass -p $password ssh root@$cam_ip "export PATH=/usr/bin:/usr/sbin:/bin:/sbin:/home/base/tools:/home/app/localbin:/home/base:/home/yi-hack-v4/bin:/home/yi-hack-v4/sbin:/tmp/sd/yi-hack-v4/bin:/tmp/sd/yi-hack-v4/sbin; export LD_LIBRARY_PATH=/usr/local/lib:/usr/lib:/home/lib:/home/hisiko/hisilib:/home/app/locallib:/home/yi-hack-v4/lib:/tmp/sd/yi-hack-v4/lib; echo `date` > /tmp/rtspv4_log 2>&1; rtspv4 >> /tmp/rtspv4_log 2>&1 &"
sshpass -p $password ssh root@$cam_ip "export PATH=/usr/bin:/usr/sbin:/bin:/sbin:/home/base/tools:/home/app/localbin:/home/base:/home/yi-hack-v4/bin:/home/yi-hack-v4/sbin:/tmp/sd/yi-hack-v4/bin:/tmp/sd/yi-hack-v4/sbin; export LD_LIBRARY_PATH=/usr/local/lib:/usr/lib:/home/lib:/home/hisiko/hisilib:/home/app/locallib:/home/yi-hack-v4/lib:/tmp/sd/yi-hack-v4/lib; echo `date` > /tmp/mqttv4_log 2>&1; mqttv4 >> /tmp/mqttv4_log 2>&1 &"

It is used the "sshpass" command (install the coresponding package for it by "sudo apt install sshpass"). At first the script kill's the rtsp server and stream daemon (sometimes it closed by himself, i d't know, why, so, i restart him "just in case") and mqtt client, and redirect the output to corresponding logs. Then it start's them in reverse order and also redirect's the outputs to logs.