search

telmomarques / xiaomi-360-1080p-hacks

Hacks for the Xiaomi Mi 360 1080p camera (MJSXJ02CM).
762 stars 129 forks source link

Find camera stream provider #2

Closed telmomarques closed 4 years ago

telmomarques commented 5 years ago

Find the stream inside the camera's system. This can be a linux device or some other (still unknown) source.

telmomarques commented 5 years ago

Here's everything I've found thus far.

The /mnt/data/bin/fetch_av binary seems to do all the heavy lifting getting a stream from the sensor, so that other binaries can stream (miio_stream), record (miio_record), etc. This seems to happen through DMA.

So, I took a look at fetch_av memory maps I've found the following interesting address mappings:

$ pidof fetch_av
198
$ cat /proc/198/maps | grep dev/mem
b52ee000-b530c000 rw-s 22cc0000 00:05 9          /dev/mem
b530c000-b5366000 rw-s 22ff0000 00:05 9          /dev/mem
b5366000-b53c0000 rw-s 23050000 00:05 9          /dev/mem
b5440000-b545e000 rw-s 22ce0000 00:05 9          /dev/mem
b545e000-b54b8000 rw-s 230b0000 00:05 9          /dev/mem
b54b8000-b5512000 rw-s 23110000 00:05 9          /dev/mem
b5592000-b565e000 rw-s 22d00000 00:05 9          /dev/mem
b565e000-b59b5000 rw-s 23230000 00:05 9          /dev/mem
b59b5000-b5d0c000 rw-s 23590000 00:05 9          /dev/mem
b5dcc000-b5e26000 rw-s 23170000 00:05 9          /dev/mem
b5e26000-b5e80000 rw-s 231d0000 00:05 9          /dev/mem
b6006000-b601d000 rw-s 22fd0000 00:05 9          /dev/mem
b62f7000-b6300000 rw-s 23df0000 00:05 9          /dev/mem
b6300000-b631c000 rw-s 22c60000 00:05 9          /dev/mem
b631c000-b6338000 rw-s 22c60000 00:05 9          /dev/mem
b6338000-b6738000 rw-s 1f000000 00:05 9          /dev/mem
b6f6a000-b6f6b000 rw-s 22c5b000 00:05 9          /dev/mem
b6f6b000-b6f6c000 rw-s 224ab000 00:05 9          /dev/mem
$ cat /proc/198/maps | grep run
b4dcf000-b4e0f000 rw-s 00000000 00:0e 16015      /run/video_mainstream/1064 (deleted)
b4e0f000-b4e30000 rw-s 00000000 00:0e 16016      /run/video_substream/1066 (deleted)
b6208000-b6209000 rw-s 00000000 00:0e 16014      /run/audio_in/1480 (deleted)

Address 0x22cc0000 is an actual image from the camera, but it's "only" a JPEG snapshot that changes every ~60 seconds. Not useful for a stream. The rest of the dump are just unidentifiable files, if anyone wants to take a look there's a download below.

The /run/video_mainstream/ and /run/video_substream/ dirs also look promising, but I still couldn't do anything with them...

extract-dmesg.zip Script to dump device DMA addresses (create memdev dir on SD Card first)

memdump.zip Complete extract-dmesg.sh dump from my camera.

telmomarques commented 5 years ago

Found another lead here, as this camera also exposes some OMX objects:

# ls /lib | grep OMX
libOMX_ACODEC.so
libOMX_ALSA.so
libOMX_ASPL.so
libOMX_AVQE_A.so
libOMX_BELA.so
libOMX_IJPE.so
libOMX_IJPE_ENC_HW.so
libOMX_IJPE_ENC_SW.so
libOMX_IJPE_MEM_MGR.so
libOMX_VMFE.so
libOMX_VSPL.so
libOMX_VSRC.so
libOMX_VVHE.so

I have been trying in the past week to grab a stream using GStreamer (with gst-omx) but I still got nothing to show. Currently trying to figure this out.

telmomarques commented 5 years ago

Gave up on trying to get something from GStreamer + OMX. Too many moving pieces, got no results from this approach.

So, I turned again to the memory dumps, played around with ffmpeg, and got a nice surprise! Turns out address 0x22CE0000 is the H.265 stream after all! Or, to be more precise, one of two available streams. Apparently there's 640x360 and 1920x1080 streams available.

I'm currently working out the details, video output from this is still very, very buggy.

DarkDenis commented 5 years ago

Hello. I understand correctly that you are close to hacking the camera and getting the rtsp stream from it?

telmomarques commented 5 years ago

I can already stream through udp using ffmpeg, but it's still really buggy. This method ("raw" direct memory access) is not perfect, but it's the only way I found until now.

Once I work out the details, the RTSP part should be easy (or at least easier).

DarkDenis commented 5 years ago

It is good news for us. You do big work. Thank you. We are waiting.

long2ice commented 5 years ago

good news!looking forward.

telmomarques commented 5 years ago

Well, I'm kinda burned out by this project... Been going at it nonstop on my spare time the past few months. I may be taking a break from it.

So, I'm sharing everything I have in this repo: https://github.com/telmomarques/xiaomi-360-1080p-video-stream

Also, here's a link to the MStar SDK for the MSC313 and MSC316: MStar MSC3XX SDK.zip

The SDK was not of much use to me, because the OS in the camera is heavily modified and the binaries produced by the SDK don't work as intended. Someone with greater experience in embedded systems may be able to do something with it, though!

I leave you with the current state of affairs:

9dragonspro commented 5 years ago

Thanks telmomarques, I follow your project https://github.com/telmomarques/xiaomi-360-1080p-video-stream and it work fine.

LeonLenclos commented 5 years ago

What you'e doing with this hack is real magic. Thanks a lot. :heart:

I've created a new folder in x360h/ (in the sd card) :

$ ls x360h/ffmpeg-video-access/
ffmpeg
ffmpeg-video-access.sh
video_direct_memory_access

with :

telmomarques commented 5 years ago

@LeonLenclos thank you. I just wish I had more time to perfect the video grabbing algorithm,

I got really demotivated because the available SDK doesn't match the camera, and doing this via DMA is like walking a knife's edge.

crckmc commented 5 years ago

Hi @telmomarques ,

first off thank you for the hard work and getting telnet access on the device. I'm currently trying to reverse engineer the libraries and binaries found on the device and found some interesting functions in "libavboard.so" like "mstar_get_videostream". Have you seen that already?

I will try to access the library later with a small tool and report if I find out anything useful.

I also found this log posted on a chinese site: https://blog.csdn.net/foxfile_hom/article/details/79125997

It looks like they wrote a tool to directly access the camera but sadly I don't speak chinese and couldn't find a way to contact the author.

Please don't give up together we will get an rtsp stream working! :)

overload08 commented 5 years ago

Hello !

May be found something :

physAddr<0x23590000> size<3502080> 1920x1088 Can you try it ?
telmomarques commented 5 years ago

@crckmc thanks for the help! Some of those functions are exposed on the MStar SDK, but unfortunately it is not 100% compatible with this camera... it seems to work up until a point, but I wasn't able to get a video stream using the SDK.

@overload08 thank you, yes that is the HD stream address :) right now the DMA proof of concept can barely handle 640x360 , so I won't be moving to a higher resolution until we can make 640x360 stable.

overload08 commented 5 years ago

@telmomarques Ok thank's Just to test, what's the value of READ_FROM_MEM_LENGTH ? I'm not developper :)

telmomarques commented 5 years ago

Hey,

The current stream grabber proof of concept is grabbing data from memory inside an infinite loop (in simple terms). READ_FROM_MEM_LENGTH is the number of bytes that we read from memory in each iteration of the cycle.

It's currently 0x7430 bytes. "0x" means it's hexadecimal, so it's 29744 bytes (or ~30KB).

crckmc commented 5 years ago

@telmomarques for easier communication I created a telegram group: https://t.me/mijav4RTSP

overload08 commented 5 years ago

I need to increase it for HD stream ?

telmomarques commented 5 years ago

I need to increase it for HD stream ?

Most likely, yes. It can be as large as the size of the allocated space on memory (3502080 bytes).

telmomarques commented 4 years ago

I'm going to abandon this approach, in favour of using a proprietary implementation found in the wild.

The proprietary implementation is not a perfect solution, but shows more potential than using DMA or the incompatible SDK.

Future updates can be found on #3.