search

fritz-smh / yi-hack

Xiaomi Yi Ants camera hack
1.39k stars 337 forks source link

stream audio to/towards the camera #58

Open eugenjeno opened 8 years ago

eugenjeno commented 8 years ago

1) how can we stream audio towards the camera from external sources. 2) how could we play an audio file g726 or wav or any format

eugenjeno commented 8 years ago

nothing?

what utility can I use to play a sound(audio file) copied to the cam. or from the SD?

vedranius commented 8 years ago

I'd like to know this too. So that we can still use two-way audio communication with Yi. As this is the main purpose of this camera! Thanks! :)

coomsie commented 7 years ago

from this pull looks like you do this ...

https://github.com/fritz-smh/yi-hack/pull/81/files

/home/rmm "/home/hd1/voice/wait.g726" 1

eugenjeno commented 7 years ago

does not work, I tried it before. When running command, the cam freezes and has to be rebooted. I get the following error: rmm_init_mpp 6604...ModId:24() is not supported !

dohlin commented 7 years ago

Anyone ever figure out anything on this? I'm new to all this stuff and trying to learn....currently using Blue Iris and neither that nor TinyCam support audio back through this cam...Really wish this could be working

wizard7926 commented 7 years ago

Also really interested in this. The RTSP streams are working well, and I like being able to cast, etc, but we're really missing the 2-way audio functionality, and I don't want to re-enable the YI app because it was opening unwelcome Play Store links on my device. Felt like spyware.

cheahkhing commented 7 years ago

same here. looking for ways to play back audio thru camera. 👍

andy2301 commented 7 years ago

@cheahkhing any findings? I tried the instructions provided in issue #178 but I couldn't even play that file in the camera.

cheahkhing commented 7 years ago

@andy2301 actually i found that i am able to play the audio file thru the camera by: For Playing back sounds thru camera: /home/rmm "/home/hd1/test/voice/wait.g726" 1 --- (for g726 file) OR /home/rmm "/home/hd1/test/voice/whatever.snd" --- (for snd file)

So, you can actually record your own sound, then convert it into the corresponding format, it should play back nicely.

eugenjeno commented 7 years ago

@cheahkhing : what"s the cmd line that you are using?

eugenjeno commented 6 years ago

@cheahkhing : what"s the cmd line that you are using?

cheahkhing commented 6 years ago

as i mentioned above, just the "/home/rmm" command eg: /home/rmm "/home/hd1/test/voice/whatever.snd"

raidenii commented 6 years ago

@cheahkhing Is the snd file Next/Sun au/snd? I tried some au/snd files but none work. Can you please post a working sample? Thanks. Also, the g726 used in firmware seems non standard, I tried to use ffprobe to detect the format but it fails.

hinbang commented 5 years ago

请问一下你找到方法了吗？最近我做的开发也涉及到了这个功能

msmarks commented 5 years ago

@hinbang recently, I decompile /home/rmm and /home/p2p_tnp and found some way to play custom audio.

/home/p2p_tnp is process handle data communicate with Mi home App /home/rmm is process to handle camera hardware device.

/home/p2p_tnp receive audio data from mi home App, then write audio data to shared memory. /home/rmm will find shared memory changed, and send audio data to hardware audio device.

I have changed the algorithm in /home/p2p_tnp. It now not read audio data from p2p network, but from file. It send audio data to shared memory and exit. Using this method, I can play g726 file, without any cam freezes.

The reason of reading audio data from file but not sending audio data to p2p_tnp by udp, because of some password in the udp payload I don't understand.

Here is the changed p2p_tnp

pgmsm.zip

you can try this program ./pgmsm /home/wait.g726

/home/rmm must run background. Because rmm is the program who play audio. pgmsm just write audio data to shared memory.

msmarks commented 5 years ago

@raidenii recently, I found some way to convert g726 file. You can try https://github.com/msmarks/python_g726, example/pcm2.py. It is a python3 script to convert pcm to g726.

First build python_g726, the run python pcm2.py wait.snd is will out put wait.g726.

the snd file can convert use some software, the param is in the pic bellow image

raidenii commented 5 years ago

thanks a lot! was trying to write a web interface to record sound and pass it to ffmpeg for transcoding. in this case then i can finish all transcoding job on backend server and just send g726 to camera. will try and update you the result!

Sent from phone

raidenii commented 5 years ago

ffmpeg can also be used for converting the file to snd: ffmpeg -i test.m4a -ar 8000 -f s16le test.snd and then use the script + library provided by @msmarks to convert snd to g726.

I tested a couple of converted g726 <10kB in size works fine. A 111kB g726 seemingly kills rmm, though...

ericjang commented 5 years ago

@msmarks Thanks for your modified pgmsm binary, but it looks like on my 47US Yi Home, its running into issues finding libstdc++.so.6, libmp4v2.so.2, and possibly other libraries. I suspect the difference is coming from different versions of base firmware on the Yi Home resulting in a dynamic linking error.

/home/yi-hack-v3/lib # /home/yi-hack-v3/bin/pgmsm /home/app/audio_file/us/success.g726 /home/yi-hack-v3/bin/pgmsm: can't load library 'libmp4v2.so.2'

Can you share the code/procedure you used to modify the p2p_tnp binary?

msmarks commented 5 years ago

@ericjang Open p2p_tnp with IDA and find do_speak in Functions window. Press F5 to translate the assemble to C. As the pic bellow, this function read the audio stream from network, by calling PPPP_Read ( Line 34）. Then put the audio stream to buffer, by calling MediaDataPut（Line 44）. image

So I changed the main function as bellow. It reads audio file then call MediaDataPut to write the buffer. image

MediaDataPut needs 4 params： first param：channel，is const 5. second param：[4 bytes header] [audio data]，in do_speaker, the header is always 2621696. I read 80 bytes audio data every loop from file. third param: data size. 4 bytes header + 80 bytes audio data = 84. fourth param: [4 bytes data size] [4 bytes timestamp] [16 bytes zeros],I guess. data size is same as third param which is 84, timestamp is the current audio data playing time, 80 bytes of g726 16bits audio data is 80 2 = 160 samples, the sample rate is 8000Hz, so duration is 160 / 8000 1000 = 20ms. the timestamp will be 0, 20, 40, 60 ... increase 20 every loop.

About editing main function. I use a stupid way:

  1. write a c file. it contains main function call MediaDataPut and empty MediaDataPut function.
  2. compile the c file to binary file by using arm-linux-uclibc-gcc. I use this project: https://buildroot.org/, here is an example to compile rsync, but in Chinese(https://www.zybuluo.com/zwh8800/note/619810)
  3. Decompile the binary file by using IDA, and find the main function.
  4. Replace the main function in p2p_tnp by my main function.
  5. Fix the address of memset, fread, MediaDataPut, usleep ...

I am a newbie in Reverse Engineering.

Piceeeeeee commented 1 month ago

@msmarks Thanks for your modified pgmsm binary, but it looks like on my 47US Yi Home, its running into issues finding libstdc++.so.6, libmp4v2.so.2, and possibly other libraries. I suspect the difference is coming from different versions of base firmware on the Yi Home resulting in a dynamic linking error.

/home/yi-hack-v3/lib # /home/yi-hack-v3/bin/pgmsm /home/app/audio_file/us/success.g726 /home/yi-hack-v3/bin/pgmsm: can't load library 'libmp4v2.so.2'

Can you share the code/procedure you used to modify the p2p_tnp binary?

Did you found a way eventually? i have the same hardware a yi home 47us