search

roleoroleo / yi-hack-Allwinner-v2

Custom firmware for Yi 1080p camera based on Allwinner platform
MIT License
844 stars 96 forks source link

Beta testing #3

Closed roleoroleo closed 3 years ago

roleoroleo commented 4 years ago

I open this issue to continue this thread https://github.com/roleoroleo/yi-hack-Allwinner/issues/157 in the right place.

roleoroleo commented 4 years ago

@lassieee Could you share the file /dev/shm/fshare_frame_buf I would like to study how to fix the stream.

lassieee commented 4 years ago

sorry for the delay.

/tmp/sd/wd_rtsp.log contains: 2020-10-03 06:34:01 - No running processes, restarting... 2020-10-03 06:34:14 - No running processes, restarting... 2020-10-03 06:34:26 - No running processes, restarting... 2020-10-03 06:34:39 - No running processes, restarting... 2020-10-03 06:34:51 - No running processes, restarting... 2020-10-03 06:35:03 - No running processes, restarting... 2020-10-03 06:35:16 - No running processes, restarting... 2020-10-03 07:02:34 - Starting RTSP watchdog... 2020-10-03 07:02:35 - No running processes, restarting... 2020-10-03 07:02:48 - No running processes, restarting... 2020-10-03 07:03:00 - No running processes, restarting... 2020-10-03 07:03:17 - Starting RTSP watchdog... 2020-10-03 07:03:18 - No running processes, restarting... 2020-10-03 07:03:30 - No running processes, restarting... 2020-10-03 07:03:43 - No running processes, restarting... 2020-10-03 07:03:55 - No running processes, restarting... 2020-10-03 07:04:07 - No running processes, restarting... 2020-10-03 07:04:20 - No running processes, restarting... 2020-10-03 07:04:32 - No running processes, restarting... 2020-10-03 07:04:45 - No running processes, restarting... 2020-10-03 07:04:57 - No running processes, restarting... 2020-10-03 07:05:09 - No running processes, restarting... 2020-10-03 07:05:22 - No running processes, restarting...

so I killed {wd_rtsp.sh} /bin/sh /tmp/sd/yi-hack/script/wd_rtsp.sh and I tried starting rRTSPServer -d 2>&1 but it quits saying port already in use. There is no other rRTSPServer process. I started rRTSPServer on port 555 in debugging, made a few connections and copied /dev/shm/fshare_frame_buf. fshare_frame_buf.tar.gz

How do you read the contents of this shm file? Do you use shmcat?

roleoroleo commented 4 years ago

This file is a custom buffer filled with the h264 stream. There is an header with a lot of unknown fields and the streams (both high and low res). If your rtsp server doesn't work, probably yi changed the structure of this file and the grabber doesn't worl correctly.

When you receive the message that the port is already in use you have to wait 20 secs. The process is terminated and it's not in memory but the socket remains in use for a long time.

I use an hex editor.

roleoroleo commented 4 years ago

fshare_frame_buf.tar.gz

The file is empty: all "00". Could you try again?

lassieee commented 4 years ago

thank you for the explanation.

I tried a couple times, but every time the file is as good as empty: $ xxd fshare_frame_buf | grep -v '0000 0000 0000 0000 0000 0000 0000 0000' 00000020: 0000 0000 0000 0000 0002 0000 0000 0000 ................ 00000030: 0000 0000 0100 0000 0000 0000 0204 0000 ................ /run/media/casper/65F0-4F4B $ xxd fshare_frame_buf_2 | grep -v '0000 0000 0000 0000 0000 0000 0000 0000' 00000020: 0000 0000 0000 0000 0002 0000 0000 0000 ................ 00000030: 0000 0000 0100 0000 0000 0000 0204 0000 ................ /run/media/casper/65F0-4F4B $ xxd fshare_frame_buf_3 | grep -v '0000 0000 0000 0000 0000 0000 0000 0000' 00000020: 0000 0000 0000 0000 0002 0000 0000 0000 ................ 00000030: 0000 0000 0100 0000 0000 0000 0204 0000 ................

I tried without intervention, I tried with killing wd_rtsp.sh, sourcing env.sh and starting wd_rtsp.sh, and I tried with running rRSTPServer -d. In each case the file was empty.

roleoroleo commented 4 years ago

How do you copy the file? Try a cp from /dev to /tmp and then ftp to transfer it to a pc.

Try also to kill rmm before the copy. Frame buffer file is filled by rmm process.

Check if rmm is running.

lassieee commented 4 years ago

I tried multiple times, copying the file to /tmp and using ftp to get it to my laptop. The result is everytime the same. When I kill rmm the camera restarts in about 5 seconds.

starting rRSTPServer and connecting with mplayer: root@yi-e209:/tmp/sd/yi-hack/bin# ./rRTSPServer -r high -a no -d debug on mapping file /dev/shm/fshare_frame_buf, size 1786156, to b6bc4000 closing the file /dev/shm/fshare_frame_buf starting capture main loop

"ch0_0.h264" stream, from memory Play this stream using the URL "rtsp://10.75.0.50/ch0_0.h264" 0 - fFrameSize 0 - fMaxSize 150000 - fLimitNumBytesToStream 0 - fPreferredFrameSize 0

$ mplayer rtsp://10.75.0.50:554/ch0_0.h264 MPlayer 1.4-10 (C) 2000-2019 MPlayer Team do_connect: could not connect to socket connect: No such file or directory Failed to open LIRC support. You will not be able to use your remote control.

Playing rtsp://10.75.0.50:554/ch0_0.h264. Resolving 10.75.0.50 for AF_INET6...

Couldn't resolve name for AF_INET6: 10.75.0.50 Connecting to server 10.75.0.50[10.75.0.50]: 554...

librtsp: server responds: '' rtsp_session: unsupported RTSP server. Server type is 'unknown'. libavformat version 58.29.100 (external) libavformat file format detected.

When starting the rtsp server it say's "closing the file /dev/shm/fshare_frame_buf". Could that be the reason it is empty?

roleoroleo commented 4 years ago

h264grabber opens the file, maps it into memory and closes it.

The problem is that fshare_frame_buf is empty while rmm is running. Does the cam work with the yi app?

lassieee commented 4 years ago

I'm not at home this week, but I took the camera with me. I think I kinda messed up. Instead of trying to change the wifi network, or making a hotspot with the same ssid and password, I pressed the reset button on the camera. I only get a blinking orange light now. On top of that, I think my sd card is acting up, and I didn't bring another. I'll see if my kids' phones have a sd card lol.

I was able to get a full boot log for the first time, with the sd card reformatted and the hack in place: minicom.log

I noticed that somewhere in the logs I saw a line that read "hit any key to stop autoboot", but for some reason it is not in the logs, and it doesn't respond to key presses.

roleoroleo commented 4 years ago

I noticed that somewhere in the logs I saw a line that read "hit any key to stop autoboot", but for some reason it is not in the logs, and it doesn't respond to key presses.

Yes, normally you have to press ENTER to stop u-boot. The TX pin of the serial port must be connected.

lassieee commented 4 years ago

Ok, that'll have to wait until I'm back home and I receive the Flux I ordered. I also found some solder with resin core so I hope I'll be able to solder the wires properly. I'll fiddle around with the camera later today. I just got new SD card so I'll see how it goes.

lassieee commented 4 years ago

I've been able to interrupt the boot proces, but I've no idea where to go from here:

Nothing updated, start normal boot Hit any key to stop autoboot: 0 => => => => helpprobe- probe sunxi card0 device sunxi_flash- sunxi_flash sub-system version - print monitor, compiler and linker version => help ? - alias for 'help' boot - boot default, i.e., run 'bootcmd' bootd - boot default, i.e., run 'bootcmd' bootm - boot application image from memory env - environment handling commands fatinfo - print information about filesystem fatload - load binary file from a dos filesystem fatls - list files in a directory (default /) fatsize - determine a file's size fatwrite- write file into a dos filesystem fdt - flattened device tree utility commands help - print command description/usage mmc - MMC sub system mmcinfo - display MMC info part_update- part_update printenv- print environment variables reset - Perform RESET of the CPU run - run commands in an environment variable saveenv - save environment variables to persistent storage setenv - set environment variables sf - SPI flash sub-system source - run script from memory sunxi_card0_probe- probe sunxi card0 device sunxi_flash- sunxi_flash sub-system version - print monitor, compiler and linker version => mmcinfo curr_device:0 Device: SUNXI SD/MMC Manufacturer ID: 2 OEM: 544d Name: SA16G Bus Speed: 25000000 Mode : SD Legacy Rd Block Len: 512 SD version 3.0 High Capacity: Yes Capacity: 14.5 GiB Bus Width: 4-bit Erase Group Size: 512 Bytes =>

It also seems /backup/init.sh on the camera is broken: /backup/init.sh: line 158: syntax error: unexpected end of file (expecting "fi")

Factory/config.sh isn't triggered to copy a good version, so at this point I don't know what to try next.

roleoroleo commented 4 years ago

It also seems /backup/init.sh on the camera is broken: /backup/init.sh: line 158: syntax error: unexpected end of file (expecting "fi")

Probably this is the problem.

Factory/config.sh isn't triggered to copy a good version, so at this point I don't know what to try next.

Follow this wiki: https://github.com/roleoroleo/yi-hack-Allwinner/wiki/Unbrick-the-cam but restore mtdblock4 and not mtdblock3. This is the mtd partition schema:

dev:    size   erasesize  name
mtd0: 00060000 00001000 "uboot"
mtd1: 001d0000 00001000 "boot"
mtd2: 00120000 00001000 "rootfs"
mtd3: 00340000 00001000 "home"
mtd4: 00130000 00001000 "backup"
mtd5: 00020000 00001000 "env"
mtd6: 00010000 00001000 "mfg"
mtd7: 00010000 00001000 "conf"
mtd8: 00000000 00000000 "UDISK"

backup partition starts from 60000 + 1d0000 + 120000 + 340000 = 690000 and the size is 1245184 = 130000 (hex)

lassieee commented 4 years ago

Thanks for the info, unfortunately I didn't make a backup :-(

roleoroleo commented 4 years ago

I will send you the file.

lassieee commented 4 years ago

I will send you the file.

Thank you, much appreciated.

roleoroleo commented 4 years ago

Here it is: mtdblock4_clean.jffs2.tar.gz

lassieee commented 4 years ago

Thanks! I have to give up for now though. I've been fiddling with my taped wires for two hours now, but the connections aren't stable. I really need to solder them. I'll be home next week to give the soldering a try.

mcanteri commented 4 years ago

Hi @roleoroleo I have the Yi cam with the same board and same firmware version as mentioned before, I can help with testing if you tell me how to go on from the point you arrived. I would like to backup the firmware before hacking it.

mcanteri commented 4 years ago

The only (minor) difference I see is on code of the processing unit, it is the same QG2101A but after I have L2010AA 6482 instead of L1026AA 63T2. Probably a different stock, later production. IMG_20201017_110955

roleoroleo commented 4 years ago

Follow this wiki to make a backup: https://github.com/roleoroleo/yi-hack-Allwinner-v2/wiki/Dump-your-backup-firmware-(SD-card)

mcanteri commented 4 years ago

Ok, perfect, manually done via telnet. I saw there are 9 block devices and actually the last one doesn't produce a file when dumped. @roleoroleo what's next ?

roleoroleo commented 4 years ago

The last mtd block is empty. Next, I need a copy of /dev/shm/fshare_frame_buf But @lassieee already tried without success: the file is zero filled.

mcanteri commented 4 years ago

I have a dump made with hexdump and another with dd, which one is the right that you want ? Mine seems not empty at all.

mcanteri commented 4 years ago

Here it is :-) fshare_frame_buf.zip

roleoroleo commented 4 years ago

Thanks.

roleoroleo commented 4 years ago

I updated the sources based on your frame buffer file -> https://github.com/roleoroleo/yi-hack-Allwinner-v2/commit/b6136e60a381c815c44ddf8dd207f6d19b8bbd2b I will post a new version asap editing this post. Please test it.

EDIT

Here it is: y21ga_0.0.4.tar.gz

mcanteri commented 4 years ago

Great, happy to be useful :+1: :-) Very interesting, so in some way in the frame buffer you can see the header size, the buffer size and offset. There is some "known" delimiter ? :)

mcanteri commented 4 years ago

BTW I have to delete all is on the SD card and untar the new hack, correct ?

roleoroleo commented 4 years ago

The frame buffer file is composed by an header and a body. The body is a circular buffer where the main process (rmm) writes h264 frames (both high res and low res, without audio). In the header there are two 4-bytes value (offset 12 and 16) that appear to be some kind of circular buffer index (the last byte written in the buffer?). The body is a sequence of h264 frames: every frame comes with a specific header (0x1c bytes long). This header contains a lot of unknown fields but I reversed these:

roleoroleo commented 4 years ago

BTW I have to delete all is on the SD card and untar the new hack, correct ?

Yes.

mcanteri commented 4 years ago

Applied and I can see the cam still works with the app and http is up, I successfully connected to http://IP:8080 :+1: I can't connect with ssh, maybe not configured to run ? Took the 2 snapshot, low and high resolution successfully. What I have to check more before to try to configure it with Home Assistant ? :)

Hostname | (none) Firmware Version | 0.0.4 Base Version | 9.0.19.02_202007141512 Model Suffix | y21ga Serial Number |   Local Time | Sun Oct 18 20:44:15 GMT 2020 Uptime | 0h 8m 53s Load Average | 1.85 1.40 0.72 Free/Total Memory | / KB Free SD Space | 99%

Missing hostname, SN and info about memory. Cannot change the root passwd in telnet: passwd: /etc/passwd: Read-only file system Status led is working, turning on when I save the configuration - before was off and I turned on Image rotation is working correctly Motion events on the page are fine, played successfully and image is good Swapon enabled and after a reboot more memory is available correctly RTSP stream is not working, I've used mplayer on linux. I have another cam and with that I successfully open the rtsp stream but not with the YI cam.

I was going to configure it with Home Assistant if RTSP was working but it didn't, so I think that is the issue for now. For what I tested it works flawless, very good software and great work !!! :+1: :) BTW hostname ant timezone and other things works, I saved the configuration and then appears correctly. :+1:

pep4n commented 4 years ago

same results here: can connect. snapshot works, stream not.

great work so far guys!

roleoroleo commented 4 years ago

ssh: found the problem, I will fix it. snapshot: excellent, it works. rtsp stream: try this new binary rRTSPServer.gz SN: know issue, please read the mmap.info file and let me know if SN is present. passwd: todo

mcanteri commented 4 years ago

Yes, with the new binary now it works! It is a bit more delayed compared to the app and when the stream starts firsts seconds aren't perfect but it seems stable. Audio is correctly streamed with video. Great progress, great work! :) 👍

I can find mmap.info file only in /tmp and it is a binary, not easily readable

lassieee commented 4 years ago

wow, great progress guys! I just restored mtdblock4 but I don't get the voice saying it's ready to connect or something to scan the qr code. I don't have wifi. In the logs I see things like: srvd: Can't open ethernet interface: wlan0 - No such device

I checked https://www.yitechnology.com/firmware/ to see if there is firmware I can install which might restore things to factory defaults, but even though the camera is listed (AI+ version), the firmware version is v8.x instead of v9.x, and the firmware is called home_y20_gam but we have y21..

Any idea's @roleoroleo , or did I properly brick it? Could it be usefull for me to restore mcanteri's backup?

putty.log

roleoroleo commented 4 years ago

@lassieee Try to reset the cam using the back button. Restart the cam without the sd.

You could try to restore mcanteri's backup but I think it's the same.

New test release: y21ga_0.0.5.tar.gz

roleoroleo commented 4 years ago

I can find mmap.info file only in /tmp and it is a binary, not easily readable

Open it with an hex editor and check if you find the serial number: something like 9FUSY21W9VL2B6100109. If you find it, I need offset and length.

mcanteri commented 4 years ago

No SN in my mmap file. I saw another little problem on the menu item "Snapshot": the link dynamically build for the page=snapshot contain the param &base64=yes that broke the image. Without that the snapshot is working.

P.S.: still using old hack, need to test it asap

roleoroleo commented 4 years ago

I don't understand. Please explain me better.

mcanteri commented 4 years ago

In the web page http://192.168.1.132:8080/?page=snapshot in the image link dynamically build there is the parameter I mentioned that broke the snapshot pic.

roleoroleo commented 4 years ago

Ok, base64 applet missing in busybox. Run this command:

cp /tmp/sd/yi-hack/bin/busybox /tmp/sd/yi-hack/bin/base64
mcanteri commented 4 years ago

busybox is not present under the folder /tmp/sd. Can you add the find command too ? Would be very helpful. Even the command free is missing, so I understand why the memory detail is missing. I tried to copy busybox from /bin/busybox to /tmp/sd/yi-hack/bin/base64 but doesn't work anyway. Checked new version, ssh now is working, great! This evening I'll do some test with HA.

I've got the DID number: dd bs=1 count=20 skip=656 if=/tmp/mmap.info 2>/dev/null -- correct option for the file status.json

roleoroleo commented 4 years ago

Sorry:

cp /tmp/sd/yi-hack/bin/gzip /tmp/sd/yi-hack/bin/base64
mcanteri commented 4 years ago

Fixed either base64 and free, now working perfectly 👍 :-)

RestOp commented 4 years ago

Subscribed

roleoroleo commented 4 years ago

Fixed either base64 and free, now working perfectly 👍 :-)

Free?!?

lassieee commented 4 years ago

wpa_supplicant.txt

@lassieee Try to reset the cam using the back button. Restart the cam without the sd.

You could try to restore mcanteri's backup but I think it's the same.

New test release: y21ga_0.0.5.tar.gz

tried that. even though I have no voice, it does try to scan the qr code, but it keeps rebooting. In the logs I notice:

/backup/tools/wpa_supplicant: line 1: y V @ V RV g▒M ▒M : not found /backup/tools/wpa_supplicant: line 2: syntax error: unexpected "(" /backup/tools/wpa_supplicant: line 1: osetsockopt_ITM_registerTMCloneTablenl_cb_allocsendmsgnl_cb_setnla_putsendtorealloc_ITM_deregisterTMCloneTablestrdupmemsetnla_nextassert_failnl_cb_errstrcmpnl_cb_put_finigenlmsg_attrlengenl_connectgenl_ctrl_resolvenlmsg_freenl_send_auto_completenl_socket_add_memberships__errno_locationatoinl_socket_alloc_cbstrlengenlmAEQA`▒rdatafputsnla_okrecvmsgfcntlregister_frame_infolibgcc_s.so.1aeabi_uldivmodabort__aeabi_idivaeabi_uidivaeabi_uidivmod__aeabi_idivmodlibc.sostrncasecmpfflushfopenfgetsvsnprintfforkhtonlreadlinkhtonsclock_gettimesetvbufgetsockoptmemmoveif_nametoindex_exitstrerrorsetsidputcharinet_ntoarecvrealpathmktimegetcwdchmodrandselectmemcmpsendfdatasyncsscanfusleepisblankgetgrnamrenamerecvfromstrchrdup2ioctlstrcasecmpvprintfsignalalphasortfputcstdoutstrncmpscandirstrtok_rchdirstrspnunlinkqsortoptargalarmstrtodstrtoulstrtolfilenorandomif_indextonamechownlocaltimegettimeofdaygmtimefseekgetoptstrstrfreadfclosemkdirftellrmdirstrrchrstatlibc_start_mainfwrite_edatabss_start__bss_start__bss_end_endendGCC_3.5▒U: Filename too long ▒▒▒▒Failed to connect to non-global ctrl_ifname: wlan0 error: No such file or directory

My wpa_supplicant file is all messed up. This should be a regular human readable file, right? wpa_supplicant.txt

Is anyone in this thread able to supply me a version of their copy of /backup/tools/wpa_supplicant?

roleoroleo commented 4 years ago

Ok try mcanteri's backup.

mcanteri commented 4 years ago

Fixed either base64 and free, now working perfectly 👍 :-)

Free?!?

yes, the command that give the memory information. Look at row 14 in status.json And at row 9 you can fix the SN with command given in the previous message. (anyway on the camera package it is mentioned as DID, not serial number and is in the form you told me BFUS... )