FS#465 - kmodloader (ubox) Illegal intruction

[aparcar]

Diffie:

Trunk builds since r3157 (after the ubox changes) produce "Illegal instruction" when kmoadloader is run on Netgear WNDR3700v4 router, resulting in no WIFI detected and no kmod-* packages being loaded at all. Manually running /sbin/kmodloader produces the above illegal intruction error.

[aparcar]

Diffie:

This is the last version that is working (git-17.026.36188-afb5c8682) / LEDE Reboot SNAPSHOT r3157+179-1f4777d20f

[aparcar]

Diffie:

Fixed this issue by enabling MIPS16 option in config file. Please close this boog!

[aparcar]

yousong:

Usually, SIGILL on MIPS was fixed by disabling MIPS16... Can you confirm that the problematic kmodloader was really compiled with mips16 disabled? It will also help if the bad binary can be provided.

[aparcar]

Diffie:

When "CONFIG_USE_MIPS16 is not set" in .config file, built package will produce illegal instruction.

With "CONFIG_USE_MIPS16=y" set in .config file the kmodloader binary will function properly.

Attached is the bad binary file, thanks!

[aparcar]

yousong:

I tried running QEMU malta-be with -cpu 74kf and gdb showed that the SIGILL was caused by a MIPS DSP instruction: lwx.

Can you also share the output of "./scripts/diffconfig.sh"?

Program received signal SIGILL, Illegal instruction. alloc_module (name=name@entry=0x414150 "af_key", aliases=aliases@entry=0x7ffffa60, naliases=naliases@entry=1, depends=, size=34600) at /mnt/storage/lede/lede/build_dir/target-mips_24kc_musl/ubox-2017-01-29-ac2d43e7/kmodloader.c:299 299 /mnt/storage/lede/lede/build_dir/target-mips_24kc_musl/ubox-2017-01-29-ac2d43e7/kmodloader.c: No such file or directory. (gdb) x/6i $pc-8 0x401d68 <alloc_module+484>: move a1,s0 0x401d6c <alloc_module+488>: addiu s1,s1,1 => 0x401d70 <alloc_module+492>: jal 0x401ac0 0x401d74 <alloc_module+496>: lwx a0,v0(a0) 0x401d78 <alloc_module+500>: b 0x401d38 <alloc_module+436> 0x401d7c <alloc_module+504>: lw v0,16(s0) (gdb)

[aparcar]

Diffie:

replied ;-)

[aparcar]

yousong:

Thanks Diffie, the diffconfig was received.

The MIPS DSP ASE was enabled by the compiler because of flag "-march=74kc". You can try adding another flag "-mno-dsp" and see if it helps

CONFIG_TARGET_OPTIMIZATION="-Os -pipe -mno-branch-likely -march=74kc -mtune=74kc"

But 74Kc should have DSP2 support and the SIGILL error should not happen. Content of /proc/cpuinfo can help.

Below is what I got with "qemu-system-mips -M malta -cpu 74Kf" and notice the dsp and dsp2 bit in "ASEs implemented".

root@LEDE:/tmp# cat /proc/cpuinfo system type : MIPS Malta machine : mti,malta processor : 0 cpu model : MIPS 74Kc V0.0 FPU V0.0 BogoMIPS : 695.09 wait instruction : yes microsecond timers : yes tlb_entries : 16 extra interrupt vector : yes hardware watchpoint : yes, count: 1, address/irw mask: [0x0ff8] isa : mips1 mips2 mips32r1 mips32r2 ASEs implemented : mips16 dsp dsp2 shadow register sets : 1 kscratch registers : 0 package : 0 core : 0 VCED exceptions : not available VCEI exceptions : not available

[aparcar]

yousong:

To debug the SIGILL, you can run gdb on your board and debug it locally, or you can use install gdbserver on your board and debug the program from your build machine. With the second option debug information on the build machine can be leveraged on to do source-level debug.

On your board

gdbserver 192.168.1.1:7001 /sbin/kmodloader

On your build machine

./scripts/remote-gdb 192.168.1.1:7001 staging_dir/target-mips_24kc_musl/root-malta/sbin/kmodloader

[aparcar]

Diffie:

Ok thanks for the lesson in gdb ;-) here's what i got:

root@router:~# cat /proc/cpuinfo system type : Atheros AR9344 rev 2 machine : NETGEAR WNDR3700v4 processor : 0 cpu model : MIPS 74Kc V4.12 BogoMIPS : 278.93 wait instruction : yes microsecond timers : yes tlb_entries : 32 extra interrupt vector : yes hardware watchpoint : yes, count: 4, address/irw mask: [0x0ffc, 0x0ffc, 0x0ffb, 0x0ffb] isa : mips1 mips2 mips32r1 mips32r2 ASEs implemented : mips16 dsp dsp2 shadow register sets : 1 kscratch registers : 0 package : 0 core : 0 VCED exceptions : not available VCEI exceptions : not available

will let you know in a bit if the "-mno-dsp" flag helps, once my build completes. Thanks!

[aparcar]

Diffie:

OK, with "CONFIG_USE_MIPS16 is not set" and "-mno-dsp" added the kmodaloder binary works fine.

[aparcar]

Diffie:

Bug is being worked on ...

[aparcar]

arjendekorte:

I'm unsure if this is a bug at all. I have a similar device (WNDR4300) and kmodloader is working fine here without any modifications (in .config, CONFIG_USE_MIPS16=y).

[aparcar]

yousong:

Default config and build (24kc and mips16) should work on this device just fine.

WNDR3700v4 is supposed to have a mips 74kc core with mips dsp2 ase extension. The issue is now that instructions like lwx should not trigger a SIGILL on this board. I am curious about the details.

[aparcar]

nbd:

Maybe QCA messed with the implementation of the CPU and removed DSP support without changing the flags responsible for detecting support for it. Either way, I'm closing this report since it was caused by an unsupported non-standard configuration