New firmware crashing

[baselsayeh]

Hello @matthiasseemoo I'm trying to port the newer 7_46_77_11 structures and offsets, but the firmware keeps crashing. And the logs are not helping too (crashes at different places and sometimes functions). Any patching with BLPatch will result with firmware crashing, but normal patches (GenericPatch4) is not affected. I'm 100% sure that the offsets are correct for the firmware, and it seems that the ucode decompress succeeds (i can see the decompressed code in the ram dump at runtime), but even simple BL patches (simple function will result in B.W) will make the firmware crash.

How can i debug this?

And here are the crash logs log 1 log 2 log 3

[baselsayeh]

Forgot to mention, Here is the simple patch i was talking about patch

And here is my work nexmon repo

[matthiasseemoo]

If you want to port nexmon to a new firmware. Try to leave out all patch files first except of patch.c This patch needs to work and extract the ucode. You will not be able to see the decompressed ucode in RAM as it is directly being decompressed into the d11 cores memory. If this simple patch is working, try adding addirional C files. If an error occurs, try to fix it.

Basel Sayeh notifications@github.com schrieb am Do., 5. Dez. 2019, 19:25:

Forgot to mention, Here is the simple patch i was talking about patch https://github.com/baselsayeh/nexmon/blob/master/patches/bcm43455/7_46_77_11_hw/nexmon/src/monitormode.c#L171

And here is my work nexmon repo https://github.com/baselsayeh/nexmon

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/nexmon/issues/352?email_source=notifications&email_token=ACZ773XN7Y35FDVXTHINWVTQXFBRXA5CNFSM4JV5USJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGBU5KY#issuecomment-562253483, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ773UUWQKY2PIMRCV2K7LQXFBRXANCNFSM4JV5USJA .

[baselsayeh]

@matthiasseemoo Yes, this is the only patch file that works, But any other patches will crash (even a simple function trampoline will result in a firmware crash).

[matthiasseemoo]

Does the basic patch really work? Can you connect to a WiFi and do you get the expected result when running nexutil -g0?

Basel Sayeh notifications@github.com schrieb am Do., 5. Dez. 2019, 20:10:

@matthiasseemoo https://github.com/matthiasseemoo Yes, this is the only patch file that works, But any other patches will crash (even a simple function trampoline will result in a firmware crash).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/nexmon/issues/352?email_source=notifications&email_token=ACZ773URGNSO7RPDGREANILQXFG3DA5CNFSM4JV5USJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGBZHDA#issuecomment-562271116, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ773XO4DOJWUSUNTBVVQ3QXFG3DANCNFSM4JV5USJA .

[baselsayeh]

@matthiasseemoo Yes ` HWEVA:/data/local/tmp # ./dhdutil membytes -r 0x198000 0xC8000 | strings | grep -i "nexmon"

pG7.46.77.11 (nexmon.org: 2.2.2-273-gaaa80e-dirty-1) Dec 5 2019 20:57:34

HWEVA:/data/local/tmp # ./nexutil -g0

0x000000: 77 6c e4 14 wl..

HWEVA:/data/local/tmp # ./nexutil -g1

0x000000: 02 00 00 00 ....

HWEVA:/data/local/tmp # ./nexutil -g2

0x000000: 00 00 00 00 ....

HWEVA:/data/local/tmp # `

[matthiasseemoo]

can you hook the ioctl function and add a simple ioctl handler, that for example, just prints something to the console.

On Thu, Dec 5, 2019 at 9:15 PM Basel Sayeh notifications@github.com wrote:

@matthiasseemoo https://github.com/matthiasseemoo Yes ` HWEVA:/data/local/tmp # ./dhdutil membytes -r 0x198000 0xC8000 | strings | grep -i "nexmon"

pG7.46.77.11 (nexmon.org: 2.2.2-273-gaaa80e-dirty-1) Dec 5 2019 20:57:34

HWEVA:/data/local/tmp # ./nexutil -g0

0x000000: 77 6c e4 14 wl..

HWEVA:/data/local/tmp # ./nexutil -g1

0x000000: 02 00 00 00 ....

HWEVA:/data/local/tmp # ./nexutil -g2

0x000000: 00 00 00 00 ....

HWEVA:/data/local/tmp # `

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/nexmon/issues/352?email_source=notifications&email_token=ACZ773WZ3GD5TDFF2AHZC4LQXFOPDA5CNFSM4JV5USJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGCAARY#issuecomment-562298951, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ773UPNXTVTOSIMD6HUN3QXFOPDANCNFSM4JV5USJA .

-- Matthias Schulz Secure Mobile Networking Lab - SEEMOO

Email: matthias.schulz@seemoo.tu-darmstadt.de Web: http://www.seemoo.de/mschulz Phone (new): +49 6151 16-25478 Fax: +49 6151 16-25471

Department of Computer Science Center for Advanced Security Research Darmstadt Technische Universität Darmstadt Mornewegstr. 32 (Office 4.2.10, Building S4/14) D-64293 Darmstadt, Germany

[baselsayeh]

@matthiasseemoo No, I can't. Any hooks other than the main patch will result in a firmware patch when using that patched function, but will continue to work fine when not using that function. (Monitor mode patch will not crash the firmware until I set the monitor option in nexutil, and ioctl will crash the firmware immediately)

[matthiasseemoo]

Good question. Did you try to disassemble or decompile the firmware and check wheter the pathes you intended to insert are working as you intended? Do you have enough free places in your flashpatching table? Maybe some structs changed that you are using. Do you have the correct driver version? Some firmware api changes sometimes require a driver update, however, if the firmware works without patches, the driver should be fine.

Basel Sayeh notifications@github.com schrieb am Fr., 6. Dez. 2019, 07:02:

@matthiasseemoo https://github.com/matthiasseemoo No, I can't. Any hooks other than the main patch will result in a firmware patch when using that patched function, but will continue to work fine when not using that function. (Monitor mode patch will not crash the firmware until I set the monitor option in nexutil, and ioctl will crash the firmware immediately)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/nexmon/issues/352?email_source=notifications&email_token=ACZ773TH5NBG33ZSLVMCACTQXHTHPA5CNFSM4JV5USJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGDDTJI#issuecomment-562444709, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ773R5VMKYBHCUNULTQRLQXHTHPANCNFSM4JV5USJA .

[baselsayeh]

@matthiasseemoo Yes, i tried Exactly what i patched:

        001ae082 7b f0 4d fc     bl         thunk_FUN_0001ed5c
...
                             thunk_FUN_0001ed5c
        00229920 f5 f5 1c ba     b.w        FUN_0001ed5c

although it looks like it will work, but it doesn't I also tried to patch a printf BL call in an unimportant function, but that also fails (works until that patched printf is called)

And the driver should not be an issue, it has all the debugging options enabled, and all other firmwares works fine.

[baselsayeh]

@matthiasseemoo Just tried to mess with the ucode stuff, and after commenting the ucode related stuff, the very simple function trampoline worked (but without any wifi functionality, because the d11 core is not initialized)

I think that either 1- the ucode portion is too big, or 2- the tinflate (deflate) stuff overwrites something important

I will look at it later tomorrow. Any suggestions?

[matthiasseemoo]

Sometimes, you need to adjust some structs to make the ucode decompression work. But so far, it either worked or it failed and you need to make it work ;-). Did you put the correct addresses into the definition.mk? May may try it without activating the flash patching.

Basel Sayeh notifications@github.com schrieb am Fr., 6. Dez. 2019, 23:12:

@matthiasseemoo https://github.com/matthiasseemoo Just tried to mess with the ucode stuff, and after commenting the ucode related stuff, the very simple function trampoline worked (but without any wifi functionality, because the d11 core is not initialized)

I think that either 1- the ucode portion is too big, or 2- the tinflate (deflate) stuff overwrites something important

I will look at it later tomorrow. Any suggestions?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/nexmon/issues/352?email_source=notifications&email_token=ACZ773V6YBJCXRF72JLECELQXLE6RA5CNFSM4JV5USJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGFPURY#issuecomment-562756167, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ773STWLI75M45UV6CVYLQXLE6RANCNFSM4JV5USJA .

[baselsayeh]

@matthiasseemoo The ucode decompress already works, but when i remove it them and implement a simple BL patches the firmware will not crash, but as soon as i re add them, that BL patch will crash the firmware when called (but will work fine if the patched function is not called), which got me confused. I'm sure that the offsets are fine, as i rechecked them 4 times.

[baselsayeh]

@matthiasseemoo And i already tried it without flash patching, but with the same result

[matthiasseemoo]

Does it work to add printf outputs to the ucode decompression function?

Basel Sayeh notifications@github.com schrieb am Sa., 7. Dez. 2019, 00:08:

@matthiasseemoo https://github.com/matthiasseemoo And i already tried it without flash patching, but with the same result

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/nexmon/issues/352?email_source=notifications&email_token=ACZ773WZDB5OG6FT36AJHK3QXLLQTA5CNFSM4JV5USJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGFUIGI#issuecomment-562775065, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ773QJUVLAAV64MFOEMETQXLLQTANCNFSM4JV5USJA .

[baselsayeh]

@matthiasseemoo Yes, but anything outside the decompression function will crash i.e: i added these printf's to help me debug things inside the decompression function

000000.002 PTRS: 0025a560 0021f9fc 54848
000000.139 Decompressed 54848 bytes!!

[matthiasseemoo]

Then try to add more printf statements in function calls directly after loading the ucode and analyze when it starts to crash exactly.

Basel Sayeh notifications@github.com schrieb am Sa., 7. Dez. 2019, 00:24:

@matthiasseemoo https://github.com/matthiasseemoo Yes, but anything outside the decompression function will crash i.e: i added these printf's to help me debug things inside the decompression function

000000.002 PTRS: 0025a560 0021f9fc 54848 000000.139 Decompressed 54848 bytes!!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/nexmon/issues/352?email_source=notifications&email_token=ACZ773R7LQMW74SAVD66GVLQXLNMFA5CNFSM4JV5USJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGFVMPA#issuecomment-562779708, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ773WFZBAZCCU5UC2J6ALQXLNMFANCNFSM4JV5USJA .

[baselsayeh]

@matthiasseemoo I'm thinking to dump whole ram memory before and after the decompression, and analyze those dumps. I will get back here if i found anything interesting.

[matthiasseemoo]

You should try to find out whether an exception is triggered that leads to printing a register and stack dump into the console ringbuffer. You can dump it as long as you still can read the RAM.

Basel Sayeh notifications@github.com schrieb am Sa., 7. Dez. 2019, 00:50:

@matthiasseemoo https://github.com/matthiasseemoo I'm thinking to dump whole ram memory before and after the decompression, and analyze those dumps. I will get back here if i found anything interesting.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/nexmon/issues/352?email_source=notifications&email_token=ACZ773UBKDJTWMFMAWGQFM3QXLQKZA5CNFSM4JV5USJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGFWZKQ#issuecomment-562785450, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ773UUMQU34P44QHAYE4DQXLQKZANCNFSM4JV5USJA .

[baselsayeh]

@matthiasseemoo I didn't know that the chip itself dump its stack in case of crashing. Anyway: that also did not help It's crashing exactly at the branch (b.w) inside the patch

000039.519 
FWID 01-543704cb
flags 1
000042.958 
TRAP 3(25fe00): pc 229920, lr 1ae087, sp 25fe58, cpsr 4000019f, spsr 400001bf
000042.958   ifsr d, ifar 229920
000042.958   r0 25a6d4, r1 25fe58, r2 2171cc, r3 0, r4 228ca4, r5 2172d0, r6 2171cc
000042.958   r7 30, r8 0, r9 2172f8, r10 14, r11 18, r12 25fed4
000042.958 
   sp+0 00000000 00000004 00301001 00000000
000042.958   sp+10 00000000 0000000e 028bf1c4 00000000

000042.958 sp+8 00301001
000042.958 sp+14 0000000e
000042.958 sp+64 001b98c9
000042.958 sp+7c 001ded9d
000042.958 sp+a4 001c37a7
000042.958 sp+dc 001ca543
000042.958 sp+104 001a947f
000042.958 sp+114 001a94bb
000042.958 sp+124 0019aa4d
000042.958 sp+128 0019b041
000042.958 sp+134 0019a127
000042.958 sp+140 000001df
000042.958 sp+180 00008221
000042.958 sp+190 0019ce0b
000042.958 sp+19c 00003531

Here is the code where it is crashing

                             thunk_FUN_0001ed5c                              XREF[1]:     FUN_001adca0:001ae082(c)  
        00229920 f5 f5 1c ba     b.w        FUN_0001ed5c

Do you know what the meaning of (3) inside the trap?

[baselsayeh]

@matthiasseemoo ifsr (d) = 0b1101, and according to This arm page, 0xd means (Permission fault on Section)

Does that mean this firmware activates the mpu? or any other protections?

[matthiasseemoo]

I have not seen it so far. However, the ucode extraction code is also placed in the memoryarea of the former ucode, hence, the area is generally executable. Try to add firmware patches in the code that is executed before the ucode extractor and check whether it leads to problems as well. Also try to place your patches in an existing function that is not used in our patch. If your patch executes there, it could be the mpu as i think that no mmu is available. You should be able to reallocate single functiony by using the attribute you also use to place patches.

Of it is the MPU oder MMU. Try to figure out where it gets configured and change the configuration to allow execution from where you place your patches.

Basel Sayeh notifications@github.com schrieb am Sa., 7. Dez. 2019, 12:46:

@matthiasseemoo https://github.com/matthiasseemoo ifsr (d) = 0b1101, and according to This arm page http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0211i/Bhcbcbbb.html, 0xd means (Permission fault on Section)

Does that mean this firmware activates the mpu? or any other protections?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/nexmon/issues/352?email_source=notifications&email_token=ACZ773RRHBJBJQQ3ZT23RNDQXOEIFA5CNFSM4JV5USJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGGFD2Q#issuecomment-562844138, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ773VKEMGY3OKEPUFVVRLQXOEIFANCNFSM4JV5USJA .

[baselsayeh]

@matthiasseemoo Finally, i found out that indeed the MPU was preventing the code from executing. and after disabling the execution protection, i can finally patch and run the firmware successfully. Still need to figure out which region is causing this issue, and try to patch that region size.

[matthiasseemoo]

Cool, it would be nice if you shared your findings in a pull request.

Basel Sayeh notifications@github.com schrieb am Sa., 7. Dez. 2019, 18:17:

Closed #352 https://github.com/seemoo-lab/nexmon/issues/352.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/nexmon/issues/352?email_source=notifications&email_token=ACZ773XDM37ITPBIQOQ733LQXPLDHA5CNFSM4JV5USJKYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOVKX6M2Y#event-2863654507, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ773SOJQLWAXQEZFRRKYTQXPLDHANCNFSM4JV5USJA .

[baselsayeh]

@matthiasseemoo I created a pull request for the patched firmwares, and i will research more later on how to change the segment's sizes in the mpu

[matthiasseemoo]

Thank you. I will review and merge it as soon as I finf some time. Dou you have a twitter account?

Basel Sayeh notifications@github.com schrieb am Sa., 7. Dez. 2019, 22:36:

@matthiasseemoo https://github.com/matthiasseemoo I created a pull request for the patched firmwares, and i will research more later on how to change the segment's sizes in the mpu

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/nexmon/issues/352?email_source=notifications&email_token=ACZ773UROKAKXYWKH5Q5FVDQXQJOLA5CNFSM4JV5USJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGGQBKI#issuecomment-562888873, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACZ773XCQOTGD26EPNCNPR3QXQJOLANCNFSM4JV5USJA .

[baselsayeh]

@matthiasseemoo Yes, i do