Ai-Thinker-Open / GPRS_C_SDK

Ai-Thinker A9/A9G GPRS (with GPS(A9G)) module C development SDK
https://ai-thinker-open.github.io/GPRS_C_SDK_DOC
MIT License
447 stars 233 forks source link

New project to develop free (as in freedom) versions of GPRS_C_SDK and CSDTK #458

Open XaviDCR92 opened 3 years ago

XaviDCR92 commented 3 years ago

Hello everyone,

As many of you might already know, Ai-Thinker has abandoned this repository and CSDTK for more than two years, with various critical issues still unresolved. So during the last few weeks I have been taking notes on how the A9, GPRS_C_SDK and CSDTK work. Then I found the firmware blobs used by the A9 that contain the bootloader, GSM/GPRS stacks, HAL and many other useful resources were released with full debug information (code was compiled using the -g flag), which makes it easier to reverse-engineer. This means it is possible to write a free (as in freedom) alternative to all the proprietary tools and firmware blobs released by Ai-Thinker that the community can benefit from.

So I have set up a repository at https://github.com/XaviDCR92/a9_free that compiles all the information I have found so far about the A9 and A9G. These are very complex pieces of hardware, so help is definitely wanted. Anyone interested in making stuff with the A9/A9G should come by and help this project becoming a reality.

So please join in our mission to free the A9 and A9G!

faithware commented 3 years ago

Thank you very much @XaviDCR92 for this. Just wanted to point that RDA8955 libraries has been published on github https://github.com/cherryding1/RDA8955_W17.44_IDH but without toolchain nor a documentation. There is so many issues in the software but the hardware is absolutely amazing.

Cheers

XaviDCR92 commented 3 years ago

Thank you very much @faithware for the link - I was not aware of it. However, most of its files are under a proprietary license (example), so unfortunately we cannot use this. Actually, I don't know where the repository author got this repository from as I cannot find any other references on the Internet. So I have written the following e-mail to Unisoc (who bought RDA Technologies) through their contact form:

Dear Unisoc Team,

I have been lately interested in one of your products, the RDA8955, which is featured on the A9 and A9G products designed by Ai-Thinker, who provide an open source toolchain (namely GPRS_C_SDK) on their Github account, allowing users to write and flash their own firmware into the microcontroller. However, they rely on a pair of proprietary binaries that contain the logic for the most important aspects of the microcontroller, such as the bootloader, the GSM/GPRS modem, the C standard library, third-party libraries and HAL. Unfortunately, the repository has been abandoned by Ai-Thinker since 2018 and there are many reported issues, some of them critical security issues, that are affected by these unmaintained binaries. We the community think the RDA8955, as well as the A9 and A9G, have great potential for many IoT solutions and deserve better. There is another Github repository, namely RDA8955_W17.44_IDH, that apparently includes the source code of these binaries. Unfortunately, the whole repository is under a proprietary license from RDA Technologies. Therefore, we the community want to solve this situation by requesting you, the copyright holder, to release all the source code from these two repositories under a free license such as the Apache, MIT, BSD or GPLv3 licenses, as well as the toolchain distributed by Ai-Thinker as CSDTK (which includes binaries of GPLv3-licensed tools modified by RDA Technologies such as the GNU toolchain, whose source must be distributed according to the license terms), so we can improve upon the existing source code and make the RDA8955 more accessible and safer against vulnerabilities. I hope you understand the benefits this brings to both you and the community. Thank you very much for your attention.

I am not sure to get any response from them, but I encourage you all to contact them so we can convince them to release their repositories under free licenses so we can maintain them. Feel free to copy the e-mail above and send it to Unisoc and Ai-Thinker.

ihewitt commented 3 years ago

I'd been trying to work out how to make calls to some of the functions that are in the elf binary but not exposed through the API. e.g. in the binary are USB host library functions. As part of that digging I came across the AliOS source which contains an RDA8955 tree: https://code.aliyun.com/alios_bull/alios/tree/dd836abd6fde8ce0a2d7cfa94cbfd4f73d970e1f/platform/mcu/rda8955 but I didn't have time to get any further working out how any of that could be built or used.

faithware commented 3 years ago

Hi guys, I think we should probably email unisoc all of us.

Cheers

XaviDCR92 commented 3 years ago

I'd been trying to work out how to make calls to some of the functions that are in the elf binary but not exposed through the API. e.g. in the binary are USB host library functions.

That should be possible by following these steps:

Now you should be able to call the function as usual.

As part of that digging I came across the AliOS source which contains an RDA8955 tree: https://code.aliyun.com/alios_bull/alios/tree/dd836abd6fde8ce0a2d7cfa94cbfd4f73d970e1f/platform/mcu/rda8955 but I didn't have time to get any further working out how any of that could be built or used.

Sounds very interesting, but it is unfortunately also released under proprietary licenses.

Hi guys, I think we should probably email unisoc all of us.

Please everyone feel free to do so - use my example mail from above if needed. I have not received any response from Unisoc or Ai-Thinker yet, but the more we contact them, the more chances we have to achieve our mission.

XaviDCR92 commented 3 years ago

I have written a tool which converts 8955_hard.xml, a file bundled with the csdtk package which describes register information, into HTML so it is more readable.

The generated HTML file is rather large (~3 MiB) so I have uploaded a compressed version here so you can read it for yourself. Some comments from the XML are still missing due to how they are laid out (see example here), but still this should help users to understand how the RDA8955 works internally.

Compressed HTML output: rda8955_register_map.zip

ihewitt commented 3 years ago

In the process of figuring out the HST protocol I've got a proof of concept code that can be used to extract flash contents directly over UART HST. It's slow (does one word at a time since I was still trying to understand how to get it working) https://github.com/ihewitt/ivrtrack/blob/main/util/dump.c

also only after writing it did I figure out which bytes are the command so it's not as generic as it should be.

XaviDCR92 commented 3 years ago

@ihewitt , I have tried your dump tool but it stops receiving at random points (less than 1000 bytes). I added the following two lines in the main loop to get progress information.

for (size_t word = 0; word < size / 4; word++) {
    printf("\r%u/%u", word, size / 4);
    fflush(stdout);

That also happens with coolwatcher sometimes. Any ideas?

ihewitt commented 3 years ago

Hmm, it was only a proof of concept, the serial receive needs completely reworking really now I know how it's arranged. It was working reliably, consistently for me. Try increasing the usleep before the read. Odd that you'd be getting that with coolwatched also though, make sure coolwatcher is disconnected when it runs. The attached will try to do nonblocking, but retry over and over on the read, give that a spin.

dump.diff.txt

XaviDCR92 commented 3 years ago

Thanks for your help! I have just realized the FTDI was connected to a USB hub shared with other peripherals and plugging the FTDI into its own port immediately solved it - no delay was needed at all, not even the usleep(700) before read on getWord. BTW, Your tool was using some C++ features that weren't mandatory, so I have slightly modified it to work with a C compiler. dump.zip

faithware commented 3 years ago

@XaviDCR92 is there any news about RDA email?

Cheers

XaviDCR92 commented 3 years ago

@faithware Unfortunately neither Ai-Thinker or Unisoc have replied so far.

XaviDCR92 commented 3 years ago

@ihewitt , I have done some experimentation with the commands used by coolwatcher and your tool and now I have a better understanding of the HST protocol. However, I still do not know how to modify the SPI flash. coolwatcher provides a clearSpiFlash command, but seems to do nothing, whereas apparently w32 can only be used for read/write memory e.g.: RAM.

devmattr commented 3 years ago

Hi, I can't even find a link to download CSDTK4.2, which I gather is the toolchain for using the c sdk. Does anyone know where I can get it? Ideally for windows, but I guess Linux would do.

Thanks!

ihewitt commented 3 years ago

@devmattr see: https://github.com/zkemble/GPRS_C_SDK/releases/tag/v2.129

ens4dz commented 2 years ago

New interesting repo: Huge code size https://github.com/ReleaseBug/RDA8955_W17.44_IDH

faithware commented 2 years ago

@ens4dz have you tried that?