riptidewave93 / LEDE-MR33

Bringup for the Cisco Meraki MR33 Access Point on LEDE
70 stars 7 forks source link

Not working on U-Boot 2017.07-RELEASE-g78ed34f31579 (Sep 29 2017 - 07:43:44 -0700) #13

Open 0xFelix opened 5 years ago

0xFelix commented 5 years ago

Seems like Cisco blocks the old method of getting a serial prompt in this uboot version?

The ubootwrite.py script is sending xyzzy, but no prompt appears.

Maybe CONFIG_AUTOBOOT_STOP_STR was changed?

Where can one obtain the latest GPL sources of Meraki's uboot?

0xFelix commented 5 years ago

Update:

I tried pressing space instead on boot... Resulted in the device saying:

Secure boot NOT enabled! Blowing fuses... Resetting now.

It is dead now, won't boot and the orange LED stays lit.

Maybe Cisco is not so keen on replacing this device's firmware afterall.

Ordspilleren commented 5 years ago

Looks like this is now reflected in the flashing documentation in Google Drive. Is there any chance this will be solved, and is there something we can do to help?

riptidewave93 commented 5 years ago

@Ordspilleren best thing to do at this point is to request the latest u-boot GPL source from Meraki. This should hopefully show us what Blowing fuses... Resetting now. actually does to the device.

Ordspilleren commented 5 years ago

I got the U-boot source from Meraki. From at quick look at the code, the whole Blowing fuses... Resetting now. ordeal seems to be related to the Qualcomm QFPROM. I am not qualified enough to figure out much more, or the solution for that matter. Here's a link for the source: https://drive.google.com/file/d/1fIs0rZI2a6UOqal6JS4nxI-Ld4co0O_B/view

argentdeux commented 5 years ago

My Meraki licence expires this year, I wait hopeful that the 2017.07 version of U-Boot is able to be flashed before then! Unfortunately this is beyond my ability.

Flole998 commented 5 years ago

Looks like what you're doing there is enabling secureboot. xyzzy should still work (with secureboot disabled) though, at least it seems like that in the code/config, I only had a brief look at the source though.

Some fuses are write once and can never be deactivated (at least not in software), I don't know if that fuse is one of those.

0xFelix commented 5 years ago

In the 20170427 u-boot sources the file include/configs/ipq40xx_cdp.h contains CONFIG_AUTOBOOT_STOP_STR set to xyzzy and in the newer u-boot sources of @Ordspilleren CONFIG_AUTOBOOT_STOP_STR is completely absent. So I guess they disabled it on purpose.

Instead the newer ipq40xx_cdp.h file defines new u-boot commands including boot_meraki_qca. This function forces secure boot and also "blew" the fuse on my device. (Looking at the code this should make no difference and it should already have been blown on my device, maybe for devices upgrading from older u-boot versions?) Devices with 20170427 u-boot seemingly do not use secure boot and are therefore able to run arbitrary code while devices with the newer u-boot only run signed code?

Richy78 commented 5 years ago

any news on the issue? I'm stuck on"uploading image" no prompt...

Flole998 commented 5 years ago

If it's really an efuse then you blow it once and that's it, no way to recover.

What you can always do when secureboot is inactive is desolder the flash and replace the uboot.

ajkenah commented 5 years ago

I found this document on Secure boot :https://www.qualcomm.com/media/documents/files/secure-boot-and-image-authentication-technical-overview.pdf Looks like secure boot has a efuse location for a root certificate hash to validate boot images. When you fail to boot a secure image it blows the fuses with the root cert.. or thats how i read the above. Not sure if Cisco even uses it though as it seems to mention that you can store certificates in ROM as well.

ajkenah commented 5 years ago

Has anyone managed to JTAG the MR33? I've got a couple of fresh ones and I would like to backup the firmware so that I can "roll-back" the Meraki bootloader later when it gets updated.. I suspect that the pinout is the 10x2 holes on the side, but I'm getting weird readings on my multi-meter and the traces don't seem to line up whats specified in the standard ARM 20-pin JTAG Layout...

satis4action commented 5 years ago

Guys, any news? My version is: U-Boot 2017.07-RELEASE-g78ed34f31579 (Sep 29 2017 - 07:43:44 -0700)

Flole998 commented 5 years ago

What did you do so far? Already pressed space and seen Secure boot NOT enabled! Blowing fuses... Resetting now.? In that case its already bricked, most likely forever. If you havent seen that yet the easiest way is most likely to replace uboot.

shanehull commented 5 years ago

any news on the issue? I'm stuck on"uploading image" no prompt...

Did you work this out?

I have the same issue. Verbose mode gives me "Waiting for prompt.." then nothing.

I've tried setting the baud rate, data bits and stop bit, but still nothing. Have verified that I have a working serial connection in/out.

ajkenah commented 5 years ago

I never got this working. I blew two rasperryPi's and then the AP itself trying to JTAG in and dump the memory so that I could flash it back if necessacary... Not that it means it doesn't work, jus tthat im not very good at soldering and double checking cable pinouts :(

owenashurst commented 4 years ago

Any update on this?

Flole998 commented 4 years ago

The method I mentioned above still works. Desolder flash, replace uboot, solder it on again.

owenashurst commented 4 years ago

The method I mentioned above still works. Desolder flash, replace uboot, solder it on again.

@Flole998 are there any instructions on this? I've never flashed a NAND flash chip before, removing it should be ok, I have a heat gun but only really done this many years ago. Also, is it the chip under the tape?

Flole998 commented 4 years ago

There are datasheets available that tell you everything you need to know about that flash. The basic steps are dump, seperate OOB/ECC, modify, calculate new ECC and write back.

What might work aswell is to short the flash when uboot tries to load the kernel, depending on it's configuration it might fall back to a console.

kitor commented 4 years ago

Was this tested anywhere?

As I'm not familiar with how wear leveling, etc works. But I have two MR33 - one running OpenWRT, one on stock software (fortunately still 2y of license to wait for any developments).

I thought about booting modded one into initrd and then hot-swapping flash and reflashing it from linux - would this work? (let's skip physical part of swapping flash in the answer)

If I understand correctly, in this kind of environment software (uboot/linux kernel) is responsible for all the leveling/ecc stuff, not memory controller like on flash drives?

Flole998 commented 4 years ago

Actually I've done it before, it's been quite some time ago though. I'm not sure if someone else has done it or if this is documented somewhere.

I've heard of hot-swapping flash causing a brick before, however that was on a different device.

What could work is cloning the other NAND if you don't want to do the modifications manually, if it contains calibration data though that would need to be written back afterwards.

owenashurst commented 4 years ago

The thing is, I've waited until the boot process finishes and I accidentally tried to exit out of screen using CTRL+C, and this gave me a Meraki> prompt. Now I don't know at which step that it causes the fuse to blow. Is it once it reboots with the uploaded openwrt u-boot and realises it's not signed or something? Because I've replicated what I've done on the same device over several reboots and I always get the prompt.

Also, I took a full back up when I successfully flashed another MR33 yesterday of MTD. Is this useful for the MR33 with the newer firmware?

satis4action commented 4 years ago

@Flole998 sorry to bothering you, can I use mr33-uboo.bin from this source? https://drive.google.com/drive/folders/1jJa8LzYnY830v3nBZdOgAk0YQK6OdbSS address should be ? 0x000000700000-0x000000900000 : "u-boot" or gerneric processes can be the same?

1.Create a full dump of the SPI Flash, and store it in a safe place
2.Erase and Clear 0x0-0x3ffff on the SPI Flash
3.Flash U-Boot to 0x0
4.Proceed to the OpenWRT Install Procedure

referrer : https://openwrt.org/toh/aruba/aruba_ap-105

Flole998 commented 4 years ago

The basic process is the same, yes. Just that this is not a SPI Flash and the address is obviously different. But dumping, replacing and writing back are the correct steps, just that in this case I desoldered the Flash because I don't want the CPU to interfere with my programming operation.

owenashurst commented 4 years ago

If you observe the boot process when you're connected via UART, it will tell you

On Wed, 15 Jan 2020, 12:27 Richard L. Alhama, notifications@github.com wrote:

hi how would you determine the uboot version without running ubootwrite.py?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/riptidewave93/LEDE-MR33/issues/13?email_source=notifications&email_token=ABBTB3CX4PL2IUH65CHMJO3Q536KTA5CNFSM4GMEISR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJAEQDI#issuecomment-574638093, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBTB3CBZO2SKL2HZVVJC5LQ536KTANCNFSM4GMEISRQ .

satis4action commented 4 years ago

connect terminal to uart port and you must see on the begining of the boot process, like this

"U-Boot 2012.07-g97ab7f1 [local,local] (Oct 06 2016 - 13:07:25)" ...

39400208-4807a69e-4b2c-11e8-8002-42f6f78b2ab6

kechie commented 4 years ago

If you observe the boot process when you're connected via UART, it will tell you On Wed, 15 Jan 2020, 12:27 Richard L. Alhama, @.***> wrote: hi how would you determine

I have U-Boot 2017.07-RELEASE-g78ed34f31579 (Sep 29 2017). Guess I have to hone my soldering skills then.

Thanks

cryptage21 commented 4 years ago

Hi guys,

Is there a way to know the bootloader version (without opening) by booting the MR33 and checking firmware version by example ?

Thanks

urbaniak commented 4 years ago

I've tried even unsoldering the flash chip, but without success - looks like it's glued and then soldered.

So the only idea I've got is clamp on tsop-48 (https://www.aliexpress.com/item/32838230005.html?spm=a2g0s.9042311.0.0.27424c4dsE1VcC) and then writing old uboot.

Flole998 commented 4 years ago

If it's glued down and they didn't put too much glue on it you can just remove it with a heatgun.

The problem with those clamps is that the CPU must be held in reset and the pins for the flash must not be pulled in either direction (low or high), otherwise the CPU will interfere and you will get weird results.

Hurricos commented 4 years ago

It appears a lot of work has been put into the process of in-circuit NAND flashing (and the documentation thereof) by Zeigren:

https://zeigren.com/portfolio/flashcatusb_clip_adapters/

Seeing as he hand assembles all of his controllers, I'm surprised he only charges $50 apiece for

Flole998 commented 4 years ago

Those are not "controllers" but simple adapters, you still need a programmer that is also capable of programming the flash.

I never tried it myself but I heard of CPUs that got fried because they tried to drive pins during programming. Just be aware that this might happen and you can end up with a CPU that'll no longer start up and gets really hot instead.

FFY00 commented 4 years ago

Just get a hot gun and take the chip out.

urbaniak commented 4 years ago

tried that, looks like it's glued to the pcb

FFY00 commented 4 years ago

At which temp, for how long? It would very weird to glue it to the pcb, even if they did that, a lot of glue can be softened by a heat gun.

Usually I use 250C, and can go up to 300C if needed. It would take a few minutes (1~3) to heat the chip at this temps, you need a little presistence :grin:.

Flole998 commented 4 years ago

That's not unusual to glue it with a few drops of glue, if you do it properly the glue will just melt. You will obviously have to pull up the chip then, that works best if you have a soldering station that uses a vacuum system to lift the chip but for this simple task even a screw driver would probably work....

urbaniak commented 4 years ago

tried two times with 300c hot air gun, solder melted but chip didn't move at all

alexsie48 commented 4 years ago

Has anyone determined the address of UBOOT. I managed to desolder the NAND and can have been able to read and dump it with a flashcat XPORT. I'm just not sure if where to find the address of UBOOT and where to write it on the flash. I could just do 0x0 and try it since I have a backup but am not sure if doing so will trip the security fuse. Here is a link to the dump as well if it helps. https://drive.google.com/file/d/11fCRvuJDIVyVaBXh9dmG4DEjKlbOS6Av/view?usp=sharing. If someone can point me in the right direction id really appreciate it.

Hurricos commented 4 years ago

Flashcat! Nice.

You want the Linux binwalk program:

$ binwalk Cypress_S34ML01G2_00-07FFFFFF.bin 

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             Qualcomm SBL1, image addr: ffffffff, image size: 4294967295, code size: 4294967295, sig size: 4294967295, cert chain size: 4294967295, oem_root_cert_sel: 4294967295, oem_num_root_certs: 4294967295
2048          0x800           Qualcomm SBL1, image addr: ffffffff, image size: 4294967295, code size: 4294967295, sig size: 4294967295, cert chain size: 4294967295, oem_root_cert_sel: 4294967295, oem_num_root_certs: 4294967295
4096          0x1000          Qualcomm SBL1, image addr: ffffffff, image size: 4294967295, code size: 4294967295, sig size: 4294967295, cert chain size: 4294967295, oem_root_cert_sel: 4294967295, oem_num_root_certs: 4294967295
6144          0x1800          Qualcomm SBL1, image addr: ffffffff, image size: 4294967295, code size: 4294967295, sig size: 4294967295, cert chain size: 4294967295, oem_root_cert_sel: 4294967295, oem_num_root_certs: 4294967295
10240         0x2800          ELF, 32-bit LSB executable, ARM, version 1 (SYSV)
98048         0x17F00         Unix path: /dev/icbcfg/boot
131072        0x20000         Qualcomm SBL1, image addr: 444d0039, image size: 808663373, code size: 1145897017, sig size: 1397555257, cert chain size: 1346961465, oem_root_cert_sel: 825242705, oem_num_root_certs: 1346961465
1052672       0x101000        ATAGs msm parition table (msmptbl), version: 4, number of paritions: 13
1183744       0x121000        ATAGs msm parition table (msmptbl), version: 4, number of paritions: 13
3145728       0x300000        ELF, 32-bit LSB executable, ARM, version 1 (SYSV)
3366555       0x335E9B        XML document, version: "1.0"
7340032       0x700000        ELF, 32-bit LSB executable, ARM, version 1 (ARM)
7464044       0x71E46C        uImage header, header size: 64 bytes, header CRC: 0xFE833387, created: 1978-10-13 04:03:19, image size: 629420935 bytes, Data Address: 0x40843387, Entry Point: 0xC8833387, data CRC: 0xA40A3487, image name: " "
7576284       0x739ADC        device tree image (dtb)
7595216       0x73E4D0        SHA256 hash constants, little endian
7613108       0x742AB4        CRC32 polynomial table, little endian
7614156       0x742ECC        CRC32 polynomial table, little endian
7621162       0x744A2A        LZO compressed data
7655600       0x74D0B0        device tree image (dtb)
12582912      0xC00000        UBI erase count header, version: 1, EC: 0x1, VID header offset: 0x800, data offset: 0x1000

Ideally, you'd find a working Meraki MR33 and copy the flash from that one to this one. This will work, because:

If the QFUSE fuse row labeled Qualcomm Secure Boot is blown (which is such on non-Chinese/OnePlus deivces), PBL (Qualcomm’s Primary Bootloader) is verified and loaded into memory from BootROM, a non-writable storgage on the SoC. PBL is then executed and brings up a nominal amount of hardware, then verifies the signature of the next bootloader in the chain, loads it, then executes it.

From https://lineageos.org/engineering/Qualcomm-Firmware/. Notice that the BootROM is not writable! So outside of the NAND you've got, there should be no record of the u-boot version.

If and when you do get a second NAND, you may want to use the Linux command vbindiff to check that the SBL1s (probably copies of U-boot bootloaders, but I'm not sure as they're quite tiny) are identical to how they are now (or at least that they sign correctly).

Hurricos commented 4 years ago

Oh, and U-boot itself is located at 0x300000. The configuration is saved much later, around 0x742AB4. Not sure why binwalk thinks it's a polynomial table.

Edit: no, it's not, as mentioned before -- it's at 0x700000. Also, based on the firmware images posted here (and this agrees with the OP), xyzzy does give console access, but apparently it then immediately checks for Secure Boot, whereas the 2016 version in this thread doesn't even mention Secure Boot.

alexsie48 commented 4 years ago

Flashcat! Nice.

You want the Linux binwalk program:

$ binwalk Cypress_S34ML01G2_00-07FFFFFF.bin 

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             Qualcomm SBL1, image addr: ffffffff, image size: 4294967295, code size: 4294967295, sig size: 4294967295, cert chain size: 4294967295, oem_root_cert_sel: 4294967295, oem_num_root_certs: 4294967295
2048          0x800           Qualcomm SBL1, image addr: ffffffff, image size: 4294967295, code size: 4294967295, sig size: 4294967295, cert chain size: 4294967295, oem_root_cert_sel: 4294967295, oem_num_root_certs: 4294967295
4096          0x1000          Qualcomm SBL1, image addr: ffffffff, image size: 4294967295, code size: 4294967295, sig size: 4294967295, cert chain size: 4294967295, oem_root_cert_sel: 4294967295, oem_num_root_certs: 4294967295
6144          0x1800          Qualcomm SBL1, image addr: ffffffff, image size: 4294967295, code size: 4294967295, sig size: 4294967295, cert chain size: 4294967295, oem_root_cert_sel: 4294967295, oem_num_root_certs: 4294967295
10240         0x2800          ELF, 32-bit LSB executable, ARM, version 1 (SYSV)
98048         0x17F00         Unix path: /dev/icbcfg/boot
131072        0x20000         Qualcomm SBL1, image addr: 444d0039, image size: 808663373, code size: 1145897017, sig size: 1397555257, cert chain size: 1346961465, oem_root_cert_sel: 825242705, oem_num_root_certs: 1346961465
1052672       0x101000        ATAGs msm parition table (msmptbl), version: 4, number of paritions: 13
1183744       0x121000        ATAGs msm parition table (msmptbl), version: 4, number of paritions: 13
3145728       0x300000        ELF, 32-bit LSB executable, ARM, version 1 (SYSV)
3366555       0x335E9B        XML document, version: "1.0"
7340032       0x700000        ELF, 32-bit LSB executable, ARM, version 1 (ARM)
7464044       0x71E46C        uImage header, header size: 64 bytes, header CRC: 0xFE833387, created: 1978-10-13 04:03:19, image size: 629420935 bytes, Data Address: 0x40843387, Entry Point: 0xC8833387, data CRC: 0xA40A3487, image name: " "
7576284       0x739ADC        device tree image (dtb)
7595216       0x73E4D0        SHA256 hash constants, little endian
7613108       0x742AB4        CRC32 polynomial table, little endian
7614156       0x742ECC        CRC32 polynomial table, little endian
7621162       0x744A2A        LZO compressed data
7655600       0x74D0B0        device tree image (dtb)
12582912      0xC00000        UBI erase count header, version: 1, EC: 0x1, VID header offset: 0x800, data offset: 0x1000

Ideally, you'd find a working Meraki MR33 and copy the flash from that one to this one. This will work, because:

If the QFUSE fuse row labeled Qualcomm Secure Boot is blown (which is such on non-Chinese/OnePlus deivces), PBL (Qualcomm’s Primary Bootloader) is verified and loaded into memory from BootROM, a non-writable storgage on the SoC. PBL is then executed and brings up a nominal amount of hardware, then verifies the signature of the next bootloader in the chain, loads it, then executes it.

From https://lineageos.org/engineering/Qualcomm-Firmware/. Notice that the BootROM is not writable! So outside of the NAND you've got, there should be no record of the u-boot version.

If and when you do get a second NAND, you may want to use the Linux command vbindiff to check that the SBL1s (probably copies of U-boot bootloaders, but I'm not sure as they're quite tiny) are identical to how they are now (or at least that they sign correctly).

Thanks for the info! I happen to have a lot of 10 of these from eBay and thankfully I have 5 with the 2016.06 Uboot and 5 with the 2017.07 so I'll pull the NAND from one of the 2016 APs and dump that. I'll definitely give those a try and hopefully, I can just overwrite the 2017 versions with the 2016 image. I'll try to upload the 2016 dump as well if anyone might want it in the future. I really appreciate the advice!

Hurricos commented 4 years ago

Wow! What luck. You may want to dump the bin of the 2016 version so we can take a look and potentially upload them to OpenWrt.org. Would be very useful for the reverse engineering process.

alexsie48 commented 4 years ago

You got it! https://drive.google.com/file/d/1QpTB6-QqDuyeMJ1ssaXxYs5zhspUaA-N/view?usp=sharing Heres the link to the 2016 NAND Dump of the mr33. Hope this helps!

satis4action commented 4 years ago

You got it! https://drive.google.com/file/d/1QpTB6-QqDuyeMJ1ssaXxYs5zhspUaA-N/view?usp=sharing Heres the link to the 2016 NAND Dump of the mr33. Hope this helps!

thank you!

riptidewave93 commented 4 years ago

If it helps, here is a dump of u-boot direct from OpenWRT, so it doesn't contain any ecc data. This is from Firmware release insect-25-201610270754 8_u-boot.zip

I also have full a full flash dump for the above firmware version, but I don't want to make it public since it contains sensitive unique-device data. Mainly, the mac address, meraki serial, and the secure token used for authenticating with Meraki to pull down it's config data.

exetico commented 4 years ago

Has anyone found a way to check the U-Boot version, without attaching wires to the UART port? Or, is this the only way to go?

antonical commented 4 years ago

Has anyone found a way to check the U-Boot version, without attaching wires to the UART port? Or, is this the only way to go?

There is no way to do this via the Meraki interface that I can find in Meraki Systems Manager.

It only takes a few minutes to open these up and connect to the UART pins. It is a very simple device to open up compared to some! Just pull off the 4 rubber feet and undo the 4 screws. Lever the case off from the sides where the backplate mounts. It will just pop off there are no wires attached to the back case. I got through 10 of these in under an hour. I used gtkterm in Ubuntu. As soon as it starts scrolling data you can disconnect and check the output. A few seconds per unit is all that is needed to see the U-Boot string.

Really hoping the wizards on this and other dev forums can find the magic string to allow flashing these U-Boot 2017.07 units as I have a pile of them waiting to go.

Cheers Tony

exetico commented 4 years ago

Good point @antonical ... And sad news, here :-)

U-Boot 2017.07-RELEASE-g78ed34f31579 (Sep 29 2017 - 07:43:44 -0700)

DRAM:  242 MiB

machid : 0x8010001

Product: meraki_Stinkbug

NAND:  ONFI device found

ID = 1d80f101

Vendor = 1

Device = f1

128 MiB

Using default environment

Thanks for the push, anyway! :-)

Hopefully someone are able to fix it, sooner or later...

antonical commented 4 years ago

The other thing I noticed is that these things are very tenacious! I had inadvertently left a MR33 on and talking to the Meraki Cloud as we were doing some testing. Even though the one I was going to flash had the correct U-Boot I had left it on for a few minutes too long and it had obviously found a connection to the Meraki Cloud via the other connected MR33. When I came to check it again it had updated to the 2017-07 release! Doh!

So be very very diligent when powering these up to ensure there is no route to the internet.

Cheers Tony

Flole998 commented 4 years ago

Hopefully someone are able to fix it, sooner or later...

No clue what you're talking about, there is already a fix mentioned since quite a while. I doubt that someone will invest more time if there is already a fix. Considering that the OpenWRT implementation is also not fully functional there's no real point in my opinion to spend more time on finding a better way to install it.