Off-cloud research

[jilleb]

@guino, I hope you don't mind me creating a seperate issue, but I think it helps to keep the other topics on-topic and easier to read.

So let's discuss here what's what when it comes to off-cloud/offline usage of the doorbell. Currently it uses the Tuya cloud, which means the following, according to @m11tch research

when internet connectivity is blocked:

• you can not view recordings from tuya app, so seems to be streaming directly from SD upon request.
• RTSP is not available. (connection refused) (will check logs later)
• snap.cgi and mjpeg.cgi work fine
• button press does make the "ding dong" sound, but no mqtt message is sent (will check logs later)
• Camera still switches to night mode if dark

[jandy123]

@jilleb Could you please fire up your IR camera ? I'm still bothered by the fact that it gets quite warm when running on 12V.

Also, do you get the "offer" to update fw. inthe Tuya app ? I'd like to bypass this bloody nag and to make sure that we won't get surprise updates from Tuya. This is next on my list...

Now, is the latest patched ppsapp working normally too ? I.e., if wan access to Tuya servers is granted ;) ?

[m11tch]

@jandy123 Sorry, work things came up, so did not get to test it earlier. used your nptpd suggestion in custom.sh and that works fine 👍

Results:

• motion detection works
• button press works
• RTSP works. seems to be even faster then before (more FPS)
• ofcourse it still tries to connect to tuya cloud endlessly.

I also get the offer about fw upgrade each time I launch the app, but at the stage we are now with the modified ppsapp I don't see a reason to allow my doorbell to connect to tuya cloud ever again ;)

[jandy123]

@m11tch

Sorry, work things came up

Well, it's about same here...

also get the offer about fw upgrade each time I launch the app, but at the stage we are now with the modified ppsapp I don't see a reason to allow my doorbell to connect to tuya cloud ever again ;)

Yep, sure, still, I'm not yet sure if I want it on/off-line yet.

[m11tch]

@jandy123 I do have an issue when viewing the RTSP stream via mplayer, after about 15 minutes the doorbell reboots. do you guys experience similar behavior?

[guino]

I am glad you guys got a handle on the time thing - I was indeed going to suggest to just set the time from the start script instead of letting ppsapp do it. My suggestion with the mqtt stuff was to modify the loop so it no longer waits instead of modifying where it connects to but whatever works for you. At some point you may need to decide between online/offline but you should have the ability to run multiple ppsapp versions from the SD card based on whatever you see fit. It maybe when you’re home you want it offline, maybe you’re traveling and want it online so you can use the app - you could make something in the start script to start the online or offline version based on something else you define (ping to tuya server, calendar information from somewhere or just a query to a local server, etc)

[jandy123]

@m11tch Well, didn't really test this. But, please do so. As I said, rtsp HD is too much for my OPI + hasssio to handle... I'll stick to the mjpeg.

@guino:

My suggestion with the mqtt stuff was to modify the loop so it no longer waits instead....

Well,, that's what I did. Just have a look :).

====

One more question: Can we bypass killing of the original ppsapp ? I'm a bit worried, if we kill it in whatever state, it may erase everything, put our house on fire, etc ;).

[jilleb]

https://imgur.com/gallery/ZJWYDVp Heat signature pictures here @jandy123

@jandy123 I do have an issue when viewing the RTSP stream via mplayer, after about 15 minutes the doorbell reboots. do you guys experience similar behavior?

It looks like it.. yes. I will do a "long term" test while being connected to my Surveillance Station and check the logs afterwards.

[jandy123]

@jileb: Thanks a lot ! Since I didn't even have to open my camera (thanks to @guino !!!), I'll have to investigate a bit which chip is what. But, maybe you guys know that already ? When on 12V, it gets hot exactly under the camera and middle, where the wires are connected. This seems to agree with your photos. There is a big with "chip" with few legs getting hot on the side, which probably is some linear regulator ??? For the rest, I couldn't spot anything unusual, right ???

Anyways, what do you guys think ?

[jandy123]

@jilleb Please test also rtsp streaming while connected to tuya servers. It could be that there is still some hidden timeout... which would be wired, since that would mean on a summer day and with no internet, under normal circumstances, the thing would get fried from so many reboots... Btw., how do we reboot the thing gracefully ???

[m11tch]

@jandy123 The crashes also occur when it is connected to cloud.. I found this out the hard way when I tested this before blocking its internet connection. (the device rebooted when I just fell asleep and the bootup melody woke me up again 😢 )

[guino]

The 15 min reboot could be the ppsapp trying to get back online - never tried to know but sounds reasonable for a cloud device. Could also be just the heat, but less likely. If you don’t want to kill ppsapp without patching the firmware you would have to basically keep a copy of the home directory in the SD card, then NOT mount the original mtdblock5 partition from flash and then edit the init.d script to NOT delete stuff after it runs, finally modify the init.d script that runs ppsapp to run your ppsapp (or just overwrite it in /home/app of the SD card). It is basically replacing the mtdblock5 partition with the SD card which is the idea of ppsAppParts=0 on boot but you would have to play with it a bit as there are some commands on the original initrun.sh that you may still need to run. You can get a copy of that initrun.sh by adding a command to your initrun.sh to copy it from the same location of app.tar.gz. Unless there’s a change in firmware version I don’t expect ppsapp to make config changes during startup.

[jandy123]

@m11tch

The crashes also occur when it is connected to cloud.

Yep, I was afraid this is the case... @guino Could we rtsp stream in SD ???

[jandy123]

@m11tch Btw. please try with the RTSP-only mod, not with "the off-line" version !!!

[m11tch]

@jandy123 my previous crashes where with the RTSP-only mod, not the offline version ;)

[guino]

You should be able to stream low res with a change in ppsapp, but before anything I would suggest checking the log file to see it says anything before rebooting - in which case we could patch that instead of changing the stream.

[jandy123]

@guino

Unless there’s a change in firmware version I don’t expect ppsapp to make config changes during startup.

Well, ppsapp does so many thing, firmware updates, resetting to factory config, etc....

[jandy123]

@guino

You should be able to stream low res with a change in ppsapp, but before anything I would suggest checking the log file to see it says anything before rebooting - in which case we could patch that instead of changing the stream.

Yep, totally agree !

@m11tch So, please try to catch the output log , maybe we can figure out what's going on. My dev. is fine in off-line mode, with no rtsp streaming ;).

[jandy123]

@guino I'll have to go back/digest over your suggestion to bypass killing ppsapp. I'll get back to this later ;).

[m11tch]

@jandy123 I'll make some changes to the log output, because currently the output file is overwritten upon reboot ;)

[jandy123]

@m11tch

I'll make some changes to the log output, because currently the output file is overwritten upon reboot ;)

Yep ;).

[m11tch]

@jandy123 okay its running with output to a different file.. I'll report back in 15 minutes or so :D

Btw my device also seems to reboot after some time if ppsapp is not running, perhaps ppsapp crashes and a few minutes later the device is rebooted.. (I'm not sure as I am not staring at the stream for 15 minutes)

[jandy123]

In the meantime a "no fw upgrade, offline" version.

ppsapp-rtsp3.zip

[jandy123]

Well, no fw up nag anymore with blocked tuya servers. Pity streaming to tuya app not working, though...

[guino]

@m11tch There's a hardware watchdog on these devices -- if it doesn't get some 'keep alive' signal the hardware reboots. I would expect ppsapp to continue to send keep alives while RTSP streaming but it is possible that the watchdog doesn't send the keep alive when the app is offline and most definitely the keep alive won't be sent when ppsapp is not running. You should not leave ppsapp stopped for long or the device will reboot (as intended by manufacturer).

[guino]

@jandy123 the tuya app live streaming works thru the cloud so you can't expect that to work without the cloud stuff working.

[m11tch]

@guino fair enough :D

stream just died again and device restarted. last log entry:

[02:35:15.631 INFO pps_watchdog.c:147] exception ocurred: 1

mplayer output:

 mplayer rtsp://192.168.2.89:8554
MPlayer 1.3.0 (Debian), built with gcc-7 (C) 2000-2016 MPlayer Team
do_connect: could not connect to socket
connect: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.

Playing rtsp://192.168.2.89:8554.
Resolving 192.168.2.89 for AF_INET6...

Couldn't resolve name for AF_INET6: 192.168.2.89
Connecting to server 192.168.2.89[192.168.2.89]: 8554...

rtsp_session: unsupported RTSP server. Server type is 'unknown'.
libavformat version 57.83.100 (external)
libavformat file format detected.
[lavf] stream 0: video (h264), -vid 0
[lavf] stream 1: audio (pcm_mulaw), -aid 0
VIDEO:  [H264]  1920x1080  0bpp  15.000 fps    0.0 kbps ( 0.0 kbyte/s)
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory
[vdpau] Error when calling vdp_device_create_x11: 1
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
libavcodec version 57.107.100 (external)
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
Clip info:
 title: tuya live stream
 comment: LIVE555 Streaming Media v2012.04.04
==========================================================================
Opening audio decoder: [alaw] aLaw/uLaw audio decoder
AUDIO: 8000 Hz, 1 ch, s16le, 64.0 kbit/50.00% (ratio: 8000->16000)
Selected audio codec: [ulaw] afm: alaw (uLaw)
==========================================================================
AO: [pulse] 8000Hz 1ch s16le (2 bytes per sample)
Starting playback...
Movie-Aspect is undefined - no prescaling applied.
VO: [xv] 1920x1080 => 1920x1080 Planar YV12 
No pts value from demuxer to use for frame!
pts after filters MISSING
A:  90.6 V:  90.6 A-V: -0.011 ct:  0.090   0/  0  3%  0%  0.2% 0 0 
[rtsp @ 0x7f12f90742a0]CSeq 8 expected, 6 received.
[rtsp @ 0x7f12f90742a0]CSeq 8 expected, 7 received.
A:  90.9 V:  92.4 A-V: -1.496 ct: -0.064   0/  0  3%  0%  0.2% 0 0 
No pts value from demuxer to use for frame!
pts after filters MISSING
A:  90.9 V:  93.7 A-V: -2.829 ct: -0.198   0/  0  3%  0%  0.2% 0 0 
[rtsp @ 0x7f12f90742a0]max delay reached. need to consume packet
[rtsp @ 0x7f12f90742a0]RTP: missed 73 packets
No pts value from demuxer to use for frame!
pts after filters MISSING
A:  94.3 V:  94.7 A-V: -0.473 ct: -0.191   0/  0  3%  0%  0.2% 0 0 
[rtsp @ 0x7f12f90742a0]max delay reached. need to consume packet
[rtsp @ 0x7f12f90742a0]RTP: missed 138 packets
A:  96.6 V:  96.3 A-V:  0.265 ct: -0.024   0/  0  3%  0%  0.2% 0 0 
[h264 @ 0x7f12f83df920]concealing 5346 DC, 5346 AC, 5346 MV errors in P frame
A:  96.6 V:  96.4 A-V:  0.239 ct: -0.018   0/  0  3%  0%  0.2% 0 0 
[h264 @ 0x7f12f83df920]Increasing reorder buffer to 1
A: 808.3 V: 808.3 A-V:  0.010 ct:  0.225   0/  0  3%  0%  0.1% 0 0 
[rtsp @ 0x7f12f90742a0]CSeq 32 expected, 9 received.
[rtsp @ 0x7f12f90742a0]CSeq 32 expected, 10 received.
A: 808.6 V: 810.0 A-V: -1.369 ct:  0.092   0/  0  3%  0%  0.1% 0 0 
[rtsp @ 0x7f12f90742a0]CSeq 32 expected, 11 received.
A: 808.6 V: 810.1 A-V: -1.435 ct:  0.078   0/  0  3%  0%  0.1% 0 0 

Exiting... (End of file)

checking the rest of the log atm but it seems to be mainly errors about connecting to cloud 😓

[jandy123]

@guino:

the tuya app live streaming works thru the cloud so you can't expect that to work without the cloud stuff working.

Yep, sure, my bad...

[jandy123]

@guino As far as I understood, rtsp streaming still leads to reboot after 15 min, even with unblocked tuya servers ???

[jandy123]

Ok, so we need to pad the dog...

[jandy123]

@m11tch Could you test (again ;) rtsp streaming with access to tuya servers ?

[m11tch]

Here is the full output log if anyone else wants to have a look.. @jandy123 sure, I will try the 'online' version in a bit.. first some food :wink:

output2.log

[guino]

From the log it does seem like the watchdog is doing the reset -- possibly because it wants the device to be back online.. but in that case I would also expect it to try and do it without using RTSP ? you should probably search for that pps_watchdog and try to figure out what exception 1 means (again just a suggestion).

[jandy123]

@guino

you should probably search for that pps_watchdog and try to figure out what exception 1 means (again just a suggestion).

Yep, it makes perfect sense, but let's just be sure what the situation is. Will test too.

[m11tch]

In the meantime a "no fw upgrade, offline" version.

[ppsapp-rtsp3.zip](https://github.com/guino/BazzDoorbell/files/5667498/ppsapp-rtsp3.zip

From the log it does seem like the watchdog is doing the reset -- possibly because it wants the device to be back online.. but in that case I would also expect it to try and do it without using RTSP ? you should probably search for that pps_watchdog and try to figure out what exception 1 means (again just a suggestion).

If I don't connect to RTSP the device does not crash/reboot or anything in offline mode. So I'm almost certain it's related to RTSP..

I'll post back in a bit with logs of online mode

[jilleb]

Log entries before my device with active RTSP feed started locking up before a reboot:

mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=1067): 0x4353C000, 8192 bytes, 'AO(0,0) CirBuf'
mmz_userdev:mmz_userdev_release: 
mmb<0x4353c000> mapped to userspace 0xb5cc4000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=1067): 0x43597000, 233472 bytes, ''
mmz_userdev:mmz_userdev_release: 
mmb<0x43597000> mapped to userspace 0xb5b7e000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=1067): 0x435D0000, 233472 bytes, ''
mmz_userdev:mmz_userdev_release: 
mmb<0x435d0000> mapped to userspace 0xb5b45000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=1067): 0x43609000, 32768 bytes, ''
mmz_userdev:mmz_userdev_release: 
mmb<0x43609000> mapped to userspace 0xb5b3d000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=1067): 0x43611000, 4096 bytes, ''
mmz_userdev:mmz_userdev_release: 
mmb<0x43611000> mapped to userspace 0xb5cc3000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=1067): 0x43613000, 479232 bytes, 'MD_ASSIST'
mmz_userdev:mmz_userdev_release: 
mmb<0x43613000> mapped to userspace 0xb5ac7000 will be unmaped!

[jandy123]

@m11tch

'll post back in a bit with logs of online mode

Yes, please do so with the rtsp-only maybe ????

[jandy123]

@jilleb Hmm, that doesn't sound right ;).

[jandy123]

Good thing is that unlocking access to tuya servers seems fine with the off-line patched version too !

Fw upgrade nag still there in app, though ;).

[m11tch]

Good thing is that unlocking access to tuya servers seems fine with the off-line patched version too !

Fw upgrade nag still there in app, though ;).

Stop using the app!! 👻

[guino]

Good thing is that unlocking access to tuya servers seems fine with the off-line patched version too ! Fw upgrade nag still there in app, though ;).

Stop using the app!! ghost

If you're going to use offline, that makes perfect sense ;)

[jilleb]

What if we make the device think it's a much higher version. Editing the version and firmware strings, then Tuya will probably think: "never mind, doesn't need an update"

[jandy123]

@jilleb

What if we make the device think it's a much higher version.

I was looking exactly into this, but couldn't figure it out. @guino Could you maybe help ;) ?

[guino]

I am pretty sure there's a structure that holds the current version, serial, etc. I think either serial or version is at offset 0x100 of the structure, but I honestly don't recall right now (I am engaged with day-work right now).

[guino]

@jandy123 if I recall correctly look at the very first function in the main sub which I think prints start information including version -- should point you to where the version is being stored.

[m11tch]

@jandy123 @jilleb

I was running ppsapp directly from command line this time, it really only prints [20:14:34.626 INFO pps_watchdog.c:147] exception ocurred: 1

before ppsapp just stops running/crashes/whatever telnet sessions stay alive, and I was also able to start ppsapp again before the device rebooted

in dmesg I'm seeing the same messages as @jilleb

mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=3492): 0x4353C000, 8192 bytes, 'AO(0,0) CirBuf'
mmz_userdev:mmz_userdev_release: 
mmb<0x4353c000> mapped to userspace 0xb5c4e000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=3492): 0x43597000, 233472 bytes, ''
mmz_userdev:mmz_userdev_release: 
mmb<0x43597000> mapped to userspace 0xb5b08000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=3492): 0x435D0000, 233472 bytes, ''
mmz_userdev:mmz_userdev_release: 
mmb<0x435d0000> mapped to userspace 0xb5acf000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=3492): 0x43609000, 32768 bytes, ''
mmz_userdev:mmz_userdev_release: 
mmb<0x43609000> mapped to userspace 0xb5ac7000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=3492): 0x43611000, 4096 bytes, ''
mmz_userdev:mmz_userdev_release: 
mmb<0x43611000> mapped to userspace 0xb5c4d000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=3492): 0x43613000, 479232 bytes, 'MD_ASSIST'
mmz_userdev:mmz_userdev_release: 
mmb<0x43613000> mapped to userspace 0xb5a51000 will be unmaped!

pid 3492 was the pid of ppsapp before it crashed.

[guino]

@m11tch so that sounds like the app is crashing for lack of cloud connection -- which explains these weird memory messages I guess. you could 'get around' it by just running ppsapp in a while loop (custom.sh) but that's kind of a rigged solution which would interrupt RTSP feed if you're looking for an always-on feed.

[m11tch]

@guino the same happens with cloud connection, it really only happens when i open the RTSP stream.

some added info, if I execute ppsapp from terminal and kill it with ctrl + c, I see the same messages in dmesg:

mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=11108): 0x4353C000, 8192 bytes, 'AO(0,0) CirBuf'
mmz_userdev:mmz_userdev_release: 
mmb<0x4353c000> mapped to userspace 0xb5ca1000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=11108): 0x43597000, 233472 bytes, ''
mmz_userdev:mmz_userdev_release: 
mmb<0x43597000> mapped to userspace 0xb5b5b000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=11108): 0x435D0000, 233472 bytes, ''
mmz_userdev:mmz_userdev_release: 
mmb<0x435d0000> mapped to userspace 0xb5b22000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=11108): 0x43609000, 32768 bytes, ''
mmz_userdev:mmz_userdev_release: 
mmb<0x43609000> mapped to userspace 0xb5b1a000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=11108): 0x43611000, 4096 bytes, ''
mmz_userdev:mmz_userdev_release: 
mmb<0x43611000> mapped to userspace 0xb5ca0000 will be unmaped!
mmz_userdev:mmz_userdev_release: 
MMB LEAK(pid=11108): 0x43613000, 479232 bytes, 'MD_ASSIST'
mmz_userdev:mmz_userdev_release: 
mmb<0x43613000> mapped to userspace 0xb5aa4000 will be unmaped!

So i'm guessing the messages in dmesg are an after effect of ppsap crashing rather then 'the source of the problem'

For what it's worth, currently capturing a log with the device in online mode and the 'original' ppsapp-rtsp

[guino]

I wouldn't be surprised if their RTSP feature is bugged and causing ppsapp to crash... I don't use RTSP enough to see if mine crashes but I can most certainly try it out to see and let you know.

[m11tch]

@m11tch so that sounds like the app is crashing for lack of cloud connection -- which explains these weird memory messages I guess. you could 'get around' it by just running ppsapp in a while loop (custom.sh) but that's kind of a rigged solution which would interrupt RTSP feed if you're looking for an always-on feed.

Then I would have to kill the startup sound ppsapp makes, cause I'm going to go mad if it's doing that every 15 minutes 😂

Looking forward to your stability of your version.. maybe it's just an issue with our version of ppsapp/our specific model?

[guino]

I don't get anything weird in dmesg, and it's been running 11 mins so far -- could also be something botched by the ppsapp mods (mine has exactly 1 byte changed).