search

guino / BazzDoorbell

124 stars 22 forks source link

4.0.7: solution(s) for my device #82

Open jonesMeUp opened 1 year ago

jonesMeUp commented 1 year ago

update: checking sd_card twice a day via crond was a bad idea. had bad weather conditions here and the doorbell takes it all as movement and soon sd-card was full. now it checks for a needed cleanup after every write

picture from my cloudfree fhem house automation: fhem-device (bell gets colored while button is down; red "led" while recording; size of streaming folder)

i have a LSC-Bell 8S (BE8S_A2_V10_433 , ppstrong-a3-tuya2_general-4.0.7.20210513) i use the device with offline option 1 and not all of your scripts worked for me. as a dump linux user, i invested a huge amount of time to get my problems solved and want to share my solution to help other people with same problems.

change values at top of initrun.sh and the private ip-range in offline.sh (next version will call offline.sh with the ip-range paramerter, so only 1 file needs to be edited)

i would appreciate every improvement from skilled linux guys...

alle needed files are located here: https://github.com/guino/BazzDoorbell/tree/master/mmc

[initrun.sh]

#!/bin/sh

MQTT_IP="192.168.178.100"      # MQTT Server IP (edit me)
MQTT_PORT="1883"               # MQTT Server Port (edit me)
DEVICE_NAME="MQTT2_DVES_DOORB" # Device Name (edit me)
MP4_Max_Size=8192 # max allowed folder size [MB] (e.g: SD-Card capacity/2)
MP4_Act_Size=-1
SERIAL=$(ppsconfig -n)
MP4_PATH="/mnt/mmc01/sdrec/T-$SERIAL/mp4rec"

#####################################################################################
# Copy app partition if not already done
#####################################################################################
if [ ! -e /mnt/mmc01/home ]; then 
 MTDNUM=`cat /proc/cmdline | sed 's/.*ppsAppParts=\([0-9]\).*/\1/'`
 mount -t cramfs /dev/mtdblock$MTDNUM /opt/pps
 tar xf /opt/pps/app.* -C /mnt/mmc01/
fi
#####################################################################################
# Update password to user setting
# (change password via 'passwd -a des <new passwd>')
#####################################################################################
cp /mnt/mmc01/passwd /etc/passwd
cp /mnt/mmc01/shadow /etc/shadow
#####################################################################################
# Start ssh server (create key is needed only once)
#####################################################################################
ln -s /mnt/mmc01/dropbearmulti /bin/scp
if [ ! -f /mnt/mmc01/dropbear_rsa_host_key ]; then 
  /mnt/mmc01/dropbearmulti dropbearkey -t rsa -f /mnt/mmc01/dropbear_rsa_host_key
fi
mkdir /etc/dropbear
cp /mnt/mmc01/dropbear_rsa_host_key /etc/dropbear/
/mnt/mmc01/dropbearmulti dropbear
#####################################################################################
# Start telnet and httpd
#####################################################################################
/mnt/mmc01/busybox telnetd
/mnt/mmc01/busybox httpd -c /mnt/mmc01/httpd.conf -h /mnt/mmc01 -p 8080
#####################################################################################
# Put the doorbell offline and redirect output from ppsapp
#####################################################################################
/mnt/mmc01/offline.sh &
#####################################################################################
# Doorbell event loop
#####################################################################################
IFS='$\n'   # Internal Field Separator
REC_TIME=-1 # -1: currently not recording
/mnt/mmc01/mqtt_pub $MQTT_IP $MQTT_PORT $DEVICE_NAME/MP4_Max_Size $MP4_Max_Size
dmesg -c > /dev/null # clear buffer
while true; do 
  BUF=$(dmesg -c | grep 'door\|motion' | grep -v 'ERR')
  ###################################################################################
  # record of motion event ended
  ###################################################################################
  if [ $REC_TIME -gt 0 ]; then let REC_TIME--; fi
  if [ $REC_TIME == 0 ]; then # record time has ended
    /mnt/mmc01/mqtt_pub $MQTT_IP $MQTT_PORT $DEVICE_NAME/recording "0"
    let REC_TIME--; 
    ###################################################################################
    # delete oldest folder (Date-PM or Date-AM) when MP4_Max_Size is reached
    ###################################################################################
    while true; do
      MP4_Act_Size=$((`/mnt/mmc01/busybox du -s "$MP4_PATH" | awk '{print $1}'` / 1024))
      /mnt/mmc01/mqtt_pub $MQTT_IP $MQTT_PORT $DEVICE_NAME/MP4_Act_Size $MP4_Act_Size
      if [ $MP4_Act_Size -lt $MP4_Max_Size ]; then break; fi;
      /mnt/mmc01/busybox find "$MP4_PATH"/* -type d | /mnt/mmc01/busybox sort | /mnt/mmc01/busybox head -n 1 | xargs rm -r
      /mnt/mmc01/mqtt_pub $MQTT_IP $MQTT_PORT $DEVICE_NAME/MP4_cleanup "done"
    done
  fi
  ###################################################################################
  # motion event
  # (getting only when ppsapp is restarted or redirected, done in offline.sh)
  ###################################################################################
  if test "${BUF#*motion detected}" != "$BUF"; then
    /mnt/mmc01/mqtt_pub $MQTT_IP $MQTT_PORT $DEVICE_NAME/recording "1"
   REC_TIME=$(grep event_record_time /home/cfg/tuya_config.json | awk '{sub(",","",$2); print $2}')
  fi
  ###################################################################################
  # doorbell event:
  # $BUF can hold >1 row, so up/down events are mostly 
  # in the same chunk. so we can use this simplification
  ###################################################################################
  if test "${BUF#*<0>door bell  press down}" != "$BUF"; then
    /mnt/mmc01/mqtt_pub $MQTT_IP $MQTT_PORT $DEVICE_NAME/state "down"
    sleep 1
    /mnt/mmc01/mqtt_pub $MQTT_IP $MQTT_PORT $DEVICE_NAME/state "up"
  fi
  ###################################################################################
  # get some sleep
  ###################################################################################
  sleep 1
done

[offline.sh]

#!/bin/sh

#DEBUG_FILE=/mnt/mmc01/offline.log
DEBUG_FILE=/dev/null
#####################################################################################
# If no hosts file, leave
#####################################################################################
if [ ! -e /mnt/mmc01/hosts ]; then
  echo "error: file '/mnt/mmc01/hosts' not found" > $DEBUG_FILE
  exit 0
fi
#####################################################################################
# Wait for the actual date (comes from tuya: so no need to check if connected)
#####################################################################################
echo "Waiting for server to send date" > $DEBUG_FILE
while [ `date +%s` -lt 1645543474 ]; do
  sleep 1
done
echo "Date was set" >> $DEBUG_FILE
#####################################################################################
# Block internet access to tuya servers (change 192.168. for your private ip range)
#####################################################################################
cp /mnt/mmc01/hosts /etc
for ip in `netstat -n 2>&1 | grep ESTABLISHED | awk '{print $5}' | awk -F: '{print $1}' | grep -v '127.0.0.1\|192.168.'`; do
  echo "checking $ip" >> $DEBUG_FILE
  if [ "`route -n | grep -c $ip`" == "0" ]; then
    route add -net $ip netmask 255.255.255.255 gw 127.0.0.1
    echo "blocked $ip" >> $DEBUG_FILE
  fi
done
echo "All tuya connections blocked, waiting until they are closed." >> $DEBUG_FILE
#####################################################################################
# WLAN off
#####################################################################################
ifconfig wlan0 down
sleep 2 # wait until WLAN is down
#####################################################################################
# Redirect output. MUST be done while WLAN is down, to get motion events from ppsapp
#####################################################################################
PPSID=$(ps | grep -v grep | grep ppsapp | awk '{print $1}')
/mnt/mmc01/reredirect -m /dev/kmsg $PPSID >> $DEBUG_FILE
#####################################################################################
# WLAN on
#####################################################################################
ifconfig wlan0 up
#####################################################################################
# Now it needs ~108sec for netstat to get rid of all tuya entries
#####################################################################################

[backup.sh]

#!/bin/sh

FILENAME="backup_$(date +%Y.%m.%d_%H%M).tar" 
rm  backup_* 2> /dev/null
(./busybox find /mnt/mmc01/ -maxdepth 1 -type f ; echo -e "/mnt/mmc01/home/\n/mnt/mmc01/cgi-bin\n/home/"; ) | ./busybox tar -cf $FILENAME -T - 
# compressing is very slow on doorbell, so perhaps comment it out
./busybox bzip2 $FILENAME

[fhem raw device data]

defmod MQTT2_DVES_DOORB MQTT2_DEVICE MQTT2_FHEM_Server
attr MQTT2_DVES_DOORB devStateIcon {\
  my $act_size = ReadingsVal($name,"MP4_Act_Size", "???");;\
  my $max_size = ReadingsVal($name,"MP4_Max_Size", "???");;\
  my $state = ReadingsVal($name,"state", "0");;\
  my $recording = ReadingsVal($name,"recording", "0");;\
  my $iconKlingel = '0';;\
  my $iconAufnahme = '0';;\
  if ($state eq "down") { $iconKlingel= '<div style="float: left"><img src="/fhem/images/default/Wecker.Wochenende.png"/></div>' }\
  else                  { $iconKlingel= '<div style="float: left"><img src="/fhem/images/default/Wecker.Aus.png"/></div>' }\
  if ($recording eq "1") { $iconAufnahme= '<div style="float: left"><img src="/fhem/images/default/15px-red.png"/></div>' }\
  else                   { $iconAufnahme= '<div style="float: left"><img src="/fhem/images/default/15px-grey.png"/></div>'}\
  return '<div>'.$iconKlingel .'<div style="float: left;; margin: 7px 15px;;">'.$iconAufnahme .'<div style="float: left;;  margin-left: 18px;;">MP4 Verzeichnis: '. $act_size.' MB / '. $max_size.' MB</div></div>'\
}
attr MQTT2_DVES_DOORB icon hue_room_frontdoor
attr MQTT2_DVES_DOORB readingList MQTT2_DVES_DOORB/state:.* state\
MQTT2_DVES_DOORB/recording:.* recording\
MQTT2_DVES_DOORB/MP4_Act_Size:.* MP4_Act_Size\
MQTT2_DVES_DOORB/MP4_Max_Size:.* MP4_Max_Size\
MQTT2_DVES_DOORB/MP4_cleanup:.* MP4_cleanup\
attr MQTT2_DVES_DOORB room MQTT2_DEVICE
guino commented 1 year ago

@jonesMeUp nice work. The only improvement I can suggest is that I just posted a method for log_parser to work without having to kill/re-run ppsapp (which can cause problems in some devices) -- that could potentially make the startup faster, but if it is working well for you (and you don't restart often) there's really no need to change it. Just in case, here's the information on getting the ppsapp output without killing/restarting it: https://github.com/guino/BazzDoorbell/issues/4#issuecomment-1208466146

jonesMeUp commented 1 year ago

ah, very cool! if there will be any ppsapp problem again, i will try your redirecting way.

it seems my ppsapp problems were:

currently i have only 1 problem left: the doorbell stream reconnect (=stuttering) depending on the running time. the longer it runs - the more it reconnects. perhaps my sd-card is too slow, perhaps its my RTSPtoWSMP4f server? hopefully, i will have a look at this problem next week. currently i am using https://github.com/deepch/RTSPtoWSMP4f (on my pi3 with fhem/ftui) because its very easy to configure and i have the stream running in the browser (pc/android) with a few lines javascript. all tries with vlc/gplayer/etc to browser and the fu*$§%# iframes were much more pain.

if there is a better/faster/easier open source way, i really want to know it...

guino commented 1 year ago

@jonesMeUp I believe there are rtsp viewer plugins you can use directly on a web page (haven't researched this in awhile), unless audio or frame rate is inportant I would just use snap/mjpeg which is what I do myself.

The connection issue you're having is likely due to crowded 2.4ghz wifi band -- there are times that it takes several seconds to get even 1 frame from my doorbell (using the phone app) because the 2.4hz wifi band here is too crowded, and if it's slow in the phone app you can be certain it's slow with everything else. Additionally my router is like 10ft from the doorbell so I know signal is not dropping.

The only thing worth mentioning is that you should be using rtsp over TCP -- the default is usually UDP and that can cause issues.

jonesMeUp commented 1 year ago

The connection issue you're having is likely due to crowded 2.4ghz wifi band

you are right, its 0:30 here and i had only 2 reconnects for some time. its not the online time of the doorbell, its a bad neighbours problem...

I believe there are rtsp viewer plugins you can use directly on a web page (haven't researched this in awhile)

negative, it isn't that easy...

unless audio or frame rate is inportant I would just use snap/mjpeg which is what I do myself.

well, framerate would be nice, audio i don't care.

The only thing worth mentioning is that you should be using rtsp over TCP

i don't have a switch for it in my current server (i have tested it on vlc a while ago, but vlc redirecting stream was still crap - i love vlc, but only on my pc)

i am testing your redirection patch atm, looks fine at first sight. i read about strace to redirect, but there wasn't a binary for my arm processor, so i was stucked to restart ppsapp. i don't want to crosscompile right now (my brain is already overloaded with the linux stuff YOU forced me in with this project)

guino commented 1 year ago

@jonesMeUp on a quick test this project worked for me for RTSP on the web browser (still requires running the RTSP client on the server): https://github.com/xpcrts/Steaming-IP-Camera-Nodejs -- this is probably similar to what you're using.

I posted the arm version or reredirect on the zip file in this post: https://github.com/guino/BazzDoorbell/issues/4#issuecomment-1208466146 it should work on all of the devices we've seen as it's compiled for armv5.

jonesMeUp commented 1 year ago

@guino reredirect seems to work fine, i have updated the files above - hoping that my solution works now for everbody... please add the reredirect file also to https://github.com/guino/BazzDoorbell/tree/master/mmc. would be nice to have all external files in 1 position.

the procejt you mentioned is ffmpeg based. i read that ffmpeg is slow and a mess if you have multiple streams (doorbell, babycamera, etc)