search

EliasKotlyar / Xiaomi-Dafang-Hacks

4.19k stars 1k forks source link

Video alerts crashing Telegram #1658

Closed devianpctek closed 3 years ago

devianpctek commented 3 years ago

Hello I'm using Dafang Hacks (7d2eeba) on a Wyze cam V2

Description

I've configure telegram to get video notifications, I get the notifications fine, but for some reason they're crashing Telegram (On Android), I tested on two different Telegram versions and Android versions, crashing on both, also this videos won't play on Telegram's video player,

jmtatsch commented 3 years ago

Telegram is very picky with respect to the encodings it displays. The current default encoding is what I got after many hours of trial and error with iOS and Mac devices. Feel free to play around with the encodings in https://github.com/EliasKotlyar/Xiaomi-Dafang-Hacks/blob/master/firmware_mod/scripts/detectionOn.sh and report back.

devianpctek commented 3 years ago

Telegram is very picky with respect to the encodings it displays. The current default encoding is what I got after many hours of trial and error with iOS and Mac devices. Feel free to play around with the encodings in https://github.com/EliasKotlyar/Xiaomi-Dafang-Hacks/blob/master/firmware_mod/scripts/detectionOn.sh and report back.

So which line/s exactly? I'm trying to understand how they work, I've changed some parameters on https://github.com/EliasKotlyar/Xiaomi-Dafang-Hacks/blob/7d2eebaed6b991a84e66bfdba807da8edcee105d/firmware_mod/scripts/detectionOn.sh#L31 and https://github.com/EliasKotlyar/Xiaomi-Dafang-Hacks/blob/7d2eebaed6b991a84e66bfdba807da8edcee105d/firmware_mod/scripts/detectionOn.sh#L247 but doesn't seems like I'm getting somewhere

jmtatsch commented 3 years ago

First you can try two different capture approaches with the motion.conf parameter video_use_rtsp=true/false . Depending on that the two lines you identified are indeed the ones to play with.

Look in the manuals for openRTSP and avconf for more command line flags to try. You can also attempt to reencode the openRTSP created file with avconv.

Here is some info what telegram expects https://webapps.stackexchange.com/questions/103895/which-formats-does-telegram-internal-video-player-support

devianpctek commented 3 years ago

First you can try two different capture approaches with the motion.conf parameter video_use_rtsp=true/false . Depending on that the two lines you identified are indeed the ones to play with.

Look in the manuals for openRTSP and avconf for more command line flags to try. You can also attempt to reencode the openRTSP created file with avconv.

Here is some info what telegram expects https://webapps.stackexchange.com/questions/103895/which-formats-does-telegram-internal-video-player-support

it seems adding -v flag (output video only) fixes the crashing issue, but still won't play in telegram's internal player. `

shiznat commented 3 years ago

Has there been any movement on this? Telegram on Android also crashes for me with video alerts so I have reverted back to using still images for the time being.

devianpctek commented 3 years ago

Has there been any movement on this? Telegram on Android also crashes for me with video alerts so I have reverted back to using still images for the time being.

Well, I tried some encodings, and change parameters, but nothing seems to work, the only "workaround" is disabling the audio, although this doesn't work either with Telegram's internal player, but at least doesn't crash when I scroll through the video notifications, this is what I'm using on /system/sdcard/scripts/detectionOn.sh /system/sdcard/bin/openRTSP -4 -B 1200000 -b 1200000 -v -V -w "$video_rtsp_w" -h "$video_rtsp_h" -f "$video_rtsp_f" -d "$video_duration" rtsp://$USERNAME:$USERPASSWORD@127.0.0.1:$PORT/unicast > "$video_tempfile"

tim-devel commented 3 years ago

I have been looking into this issue and I think I have found the root cause - the audio codec. If you set audio to none the following patch in detectionOn.sh will send a video that plays in the telegram android app.

/system/sdcard/bin/avconv -i "$video_tempfile" -c copy "$video_tempfile"-telegram.mp4
/system/sdcard/bin/telegram v "$video_tempfile"-telegram.mp4
rm "$video_tempfile"-telegram.mp4

If you set audio to mp3 the rtsp capture binary complains this is out of spec and throws this error on each capture:

Warning: We don't implement a QuickTime Audio Media Data Type for the "MPA" track, so we'll insert a dummy "????" Media Data Atom instead.  A separate, codec-specific editing pass will be needed before this track can be played.

This makes a file a video file with a dodgy header so it is unplayable and avconv fails by default. I have had some more success using PCMU audio codec as rtsp accepts this format but files still don't play properly in telegram.

Having done some googling I think telegram needs AAC audio but the avconv shipped with dafang hacks only had experimental sorry for AAC and fails to convert the PCMU to AAC. I am going to try and compile a recent version of ffmeg with AAC support and will report back.

tim-devel commented 3 years ago

Have done some more testing. If you move the h264 video file to a Linux pc/laptop, run the command below, copy the new file back to the camera and send via telegram the video file plays perfectly in the telegram app. You have to set audio to PCMU in camera settings as other settings do not work.

ffmpeg -i video.mp4 -c:v copy -c:a aac telegram_video.mp4

The problem is the version of avconv shipped with the camera is ancient and only has an experimental version of the AAC encoder. I have tried to get it to transcode the file to AAC but it won't work. As stated in this stack exchange question we need to update libav/ffmpeg - https://stackoverflow.com/questions/10330215/ffmpeg-too-many-bits-per-frame-requested. I have been trying to compile a recent version of ffmpeg for MIPS but have so far failed.

To get the videos to play in telegram natively with audio we will either need to compile a newer version of ffmpeg (I have tried and failed) or add an AAC encoder to the rtsp-server (well beyond my level of skill).

jmtatsch commented 3 years ago

@timstanley1985 where are you stuck compiling avconv? maybe open a WIP PR at the https://github.com/Dafang-Hacks/Main repository so others can have a look and pitch in

tim-devel commented 3 years ago

No problem, I'll tidy up my compile script and do a WIP PR later next week. I have also spotted a problem with openRTSP that is linked to these issues, will write a longer post on that in next few days

tim-devel commented 3 years ago

@timstanley1985 where are you stuck compiling avconv? maybe open a WIP PR at the https://github.com/Dafang-Hacks/Main repository so others can have a look and pitch in

Dafang-Hacks/Main#85

tim-devel commented 3 years ago

According to the Telegram spec it will only accept a file in an MP4 container with audio encoded in AAC. According to the docs for openRTSP (http://www.live555.com/openRTSP/#quicktime) the -4 flag encodes the file as MP4 and -q encodes as Quicktime. I have been doing some testing and even though we are using the -4 flag the file is being encoded into a Quicktime container. This is why the avconv command is required as this copies the video into a proper MP4 container which then plays in Telegram.

I am going to try and compile a new version of openRTSP with a working -4 flag and will post a WIP to dafang hack main.

I have pasted some shell output below and highlighted some key points:

[root@KitchenCam:~]# openRTSP -4 -w 1920 -h 1080 -f 10 -d 10 -b 1500000 rtsp://127.0.0.1:8554/unicast> test.mp4
Created new TCP socket 3 for connection
Connecting to 127.0.0.1, port 8554 on socket 3...
...remote connection opened
Sending request: OPTIONS rtsp://127.0.0.1:8554/unicast RTSP/1.0
CSeq: 2
User-Agent: openRTSP (LIVE555 Streaming Media v2020.05.15)

Received 152 new bytes of response data.
Received a complete OPTIONS response:
RTSP/1.0 200 OK
CSeq: 2
Date: Mon, Mar 15 2021 19:28:09 GMT
Public: OPTIONS, DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE, GET_PARAMETER, SET_PARAMETER

Sending request: DESCRIBE rtsp://127.0.0.1:8554/unicast RTSP/1.0
CSeq: 3
User-Agent: openRTSP (LIVE555 Streaming Media v2020.05.15)
Accept: application/sdp

Received 709 new bytes of response data.
Received a complete DESCRIBE response:
RTSP/1.0 200 OK
CSeq: 3
Date: Mon, Mar 15 2021 19:28:09 GMT
Content-Base: rtsp://127.0.0.1:8554/unicast/
Content-Type: application/sdp
Content-Length: 546

v=0
o=- 1615722675176757 1 IN IP4 10.10.10.239
s=LIVE555 Streaming Media v2020.05.15
i=LIVE555 Streaming Media v2020.05.15
t=0 0
a=tool:LIVE555 Streaming Media v2020.05.15
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:LIVE555 Streaming Media v2020.05.15
a=x-qt-text-inf:LIVE555 Streaming Media v2020.05.15
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:10
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=4d4028;sprop-parameter-sets=Z01AKJ2oHgCJ+WEAAAMAAQAAAwAyhA==,aO48gA==
a=x-dimensions:1920,1080
a=control:track1

Opened URL "rtsp://127.0.0.1:8554/unicast", returning a SDP description:
v=0
o=- 1615722675176757 1 IN IP4 10.10.10.239
s=LIVE555 Streaming Media v2020.05.15
i=LIVE555 Streaming Media v2020.05.15
t=0 0
a=tool:LIVE555 Streaming Media v2020.05.15
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:LIVE555 Streaming Media v2020.05.15
a=x-qt-text-inf:LIVE555 Streaming Media v2020.05.15
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:10
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=4d4028;sprop-parameter-sets=Z01AKJ2oHgCJ+WEAAAMAAQAAAwAyhA==,aO48gA==
a=x-dimensions:1920,1080
a=control:track1

Created receiver for "video/H264" subsession (client ports 56778-56779)
Sending request: SETUP rtsp://127.0.0.1:8554/unicast/track1 RTSP/1.0
CSeq: 4
User-Agent: openRTSP (LIVE555 Streaming Media v2020.05.15)
Transport: RTP/AVP;unicast;client_port=56778-56779

Received 208 new bytes of response data.
Received a complete SETUP response:
RTSP/1.0 200 OK
CSeq: 4
Date: Mon, Mar 15 2021 19:28:09 GMT
Transport: RTP/AVP;unicast;destination=127.0.0.1;source=127.0.0.1;client_port=56778-56779;server_port=6970-6971
Session: 1F2DC088;timeout=65

Setup "video/H264" subsession (client ports 56778-56779)
Outputting to the file: "stdout"
Sending request: PLAY rtsp://127.0.0.1:8554/unicast/ RTSP/1.0
CSeq: 5
User-Agent: openRTSP (LIVE555 Streaming Media v2020.05.15)
Session: 1F2DC088
Range: npt=0.000-10.000

Received 184 new bytes of response data.
Received a complete PLAY response:
RTSP/1.0 200 OK
CSeq: 5
Date: Mon, Mar 15 2021 19:28:09 GMT
Range: npt=0.000-
Session: 1F2DC088
RTP-Info: url=rtsp://127.0.0.1:8554/unicast/track1;seq=45874;rtptime=3295408827

Started playing session
Receiving streamed data (for up to 10.000000 seconds)...
Sending request: TEARDOWN rtsp://127.0.0.1:8554/unicast/ RTSP/1.0
CSeq: 6
User-Agent: openRTSP (LIVE555 Streaming Media v2020.05.15)
Session: 1F2DC088

Received 65 new bytes of response data.
Received a complete TEARDOWN response:
RTSP/1.0 200 OK
CSeq: 6
Date: Mon, Mar 15 2021 19:28:19 GMT

[root@KitchenCam:~]# mediainfo test.mp4 
General
Complete name                            : test.mp4
Format                                   : QuickTime   <<<<<<<< FILE IS IN QUICKTIME FORMAT EVEN THOUGH WE SET -4 FLAG
Format/Info                              : Original Apple specifications
File size                                : 1.18 MiB
Duration                                 : 12 s 700 ms
Overall bit rate                         : 778 kb/s
Encoded date                             : UTC 2021-03-15 19:28:09
Tagged date                              : UTC 2021-03-15 19:28:09
Writing library                          : Apple QuickTime
FileExtension_Invalid                    : braw mov qt

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Main@L4
Format settings                          : CABAC / 1 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 1 frame
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 12 s 700 ms
Bit rate                                 : 777 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 10.000 FPS
Original frame rate                      : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.037
Stream size                              : 1.18 MiB (100%)
Language                                 : English
Encoded date                             : UTC 2021-03-15 19:28:09
Tagged date                              : UTC 2021-03-15 19:28:09
Codec configuration box                  : avcC

[root@KitchenCam:~]# avconv -i test.mp4 -c:v copy -c:a copy test2.mp4
avconv version v13_dev0-1541-ge368b0cbf, Copyright (c) 2000-2018 the Libav developers
  built on May 17 2018 20:39:55 with gcc 4.7.2 (Ingenic r2.3.3 2016.12)
[h264 @ 0x18e5ee0] no frame!
    Last message repeated 3 times
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42isom
    creation_time   : 2021-03-15 19:28:09
  Duration: 00:00:12.70, start: 0.000000, bitrate: 778 kb/s
    Stream #0:0(eng): Video: h264 (Main) [avc1 / 0x31637661]
      yuv420p, 1920x1080, 776 kb/s
      10 fps, 600 tbn (default)
    Metadata:
      creation_time   : 2021-03-15 19:28:09
      handler_name    : Apple Alias Data Handler
      encoder         : H.264
Output #0, mp4, to 'test2.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42isom
    creation_time   : 2021-03-15 19:28:09
    encoder         : Lavf58.2.0
    Stream #0:0(eng): Video: h264 (Main) [avc1 / 0x31637661]
      yuv420p, 1920x1080, q=2-31, 776 kb/s
      600 tbn (default)
    Metadata:
      creation_time   : 2021-03-15 19:28:09
      handler_name    : Apple Alias Data Handler
      encoder         : H.264
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press ctrl-c to stop encoding
frame=  127 fps=  0 q=-1.0 Lsize=    1206kB time=12.60 bitrate= 784.3kbits/s    
video:1205kB audio:0kB other streams:0kB global headers:0kB muxing overhead: 0.150714%
[root@KitchenCam:~]# mediainfo test2.mp4 
General
Complete name                            : test2.mp4
Format                                   : MPEG-4  <<<<<<<<<<AFTER AVCONV FILE IS IN PROPER MP4 CONTAINER
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 1.18 MiB
Duration                                 : 12 s 700 ms
Overall bit rate                         : 778 kb/s
Encoded date                             : UTC 2021-03-15 19:28:09
Tagged date                              : UTC 2021-03-15 19:28:09
Writing application                      : Lavf58.2.0

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Main@L4
Format settings                          : CABAC / 1 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 1 frame
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 12 s 700 ms
Bit rate                                 : 777 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 10.000 FPS
Original frame rate                      : 25.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.037
Stream size                              : 1.18 MiB (100%)
Language                                 : English
Encoded date                             : UTC 2021-03-15 19:28:09
Tagged date                              : UTC 2021-03-15 19:28:09
Codec configuration box                  : avcC

[root@KitchenCam:~]#
tim-devel commented 3 years ago

I think the ffmpeg option is a no-go. I used the instructions in this issue to install entware onto the camera EliasKotlyar/Xiaomi-Dafang-Hacks#542

There is a quite an old version of ffmpeg you can install (from 2017) with a working AAC encoder but it is painfully slow. Transcoding a 10 second video took 90 secs.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.