search

Mr-Bossman / pi-pico2-linux

RISC-V linux on the raspberry pi pico 2
135 stars 11 forks source link

test_executability fails #1

Closed arturo182 closed 1 week ago

arturo182 commented 3 weeks ago

I'm trying to get the setup running, my PSRAM seems to initialize correctly and I have tested it previously with CircuitPython, but https://github.com/Mr-Bossman/pi-pico2-linux/blob/master/psram-bootloader/src/bootloader.c#L187 results in a HardFault, any advice on what I could look into to get it working?

PS. GPIO0 is a valid XIP_CS and https://github.com/Mr-Bossman/pi-pico2-linux/blob/master/psram-bootloader/src/bootloader.c#L274 is stopping it from being accepted :grin:

Mr-Bossman commented 3 weeks ago

Oh, sorry, I missed this. We'll nice to know my check executableity function works correctly. Do u have a screenshot of the output. And what board are you using. Speaking of which, I should probably add an image to the readme.

arturo182 commented 3 weeks ago

Im afk right now so don't have the output, but I had the same result (HardFault as seen in gdb) on https://shop.pimoroni.com/products/Pimoroni-Pico-plus-2, https://ilabs.se/product/challenger-rp2350-wifi-ble/, and https://www.lectronz.com/products/rp2350-stamp-xl

Mr-Bossman commented 3 weeks ago

the https://ilabs.se/product/challenger-rp2350-wifi-ble has the same psram chip the other ones dont. so it should be as ez as changing SFE_RP2350_XIP_CSI_PIN to 0

arturo182 commented 2 weeks ago

Both the Sparkfun Pro Micro and the Pimoroni Pico Plus 2 use the APS6404L, but the Pimoroni one is in a smaller package. On the Stamp, I use an ESP-PSRAM64, so that is different. The only change for Pimoroni and iLabs was to adjust the PSRAM pin.

Here is a screenshot of the serial and gdb output when running on the Pimoroni Pico Plus 2 (same result as the iLabs): image

Mr-Bossman commented 2 weeks ago

Both the Sparkfun Pro Micro and the Pimoroni Pico Plus 2 use the APS6404L, but the Pimoroni one is in a smaller package.

Ah ok.

On the Stamp, I use an ESP-PSRAM64, so that is different. The only change for Pimoroni and iLabs was to adjust the PSRAM pin.

It seems like writing to the ram is working, as in R/W, but it isn't executing. I'm assuming you have changed the board here https://github.com/Mr-Bossman/pi-pico2-linux/blob/master/psram-bootloader/CMakeLists.txt#L37. I honestly have no clue why it doesn't work, I would just play around with it. I'm on vacation with my friends so cant be of help for the next few days, though here is what the output is supposed to look like.

image

arturo182 commented 2 weeks ago

No worries, enjoy your vacation and write back whenever you have time. You did amazing work on the port already! Thanks for trying to help me, I'll try to poke some things and see if I can figure it out :)

arturo182 commented 2 weeks ago

Here's a dump of all the registers:

(gdb) info registers 
r0             0x1d                29
r1             0x200019b0          536877488
r2             0x0                 0
r3             0x8082              32898
r4             0x117ffffc          293601276
r5             0x2000199c          536877468
r6             0x8082              32898
r7             0x400e0014          1074659348
r8             0x43280035          1126694965
r9             0x0                 0
r10            0x10000000          268435456
r11            0x62707361          1651536737
r12            0xff                255
sp             0x20081fc8          0x20081fc8
lr             0xfffffff9          -7
pc             0x10000114          0x10000114 <isr_hardfault>
xpsr           0x69000003          1761607683
fpscr          0x0                 0
msp            0x20081fc8          0x20081fc8
psp            0x0                 0x0
msp_ns         0x0                 0x0
psp_ns         0xfffffffc          0xfffffffc
msp_s          0x20081fc8          0x20081fc8
psp_s          0x0                 0x0
primask        0x0                 0
basepri        0x0                 0
faultmask      0x0                 0
control        0x0                 0
msplim_s       0x0                 0x0
psplim_s       0x0                 0x0
msplim_ns      0x0                 0x0
psplim_ns      0x0                 0x0
primask_s      0x0                 0
basepri_s      0x0                 0
faultmask_s    0x0                 0
control_s      0x0                 0
primask_ns     0x0                 0
basepri_ns     0x0                 0
faultmask_ns   0x0                 0
control_ns     0x0                 0

Reading the UsageFault Status Register (UFSR), we get 0x2 = INVSTATE

(gdb) print/x *(uint16_t *)0xE000ED2A
$1 = 0x2

Which according to https://interrupt.memfault.com/blog/cortex-m-hardfault-debug#ufsr means

INVSTATE - Indicates the processor has tried to execute an instruction with an invalid Execution Program Status Register (EPSR) value. Among other things the ESPR tracks whether or not the processor is in thumb mode state. Instructions which use “interworking addresses”2 (bx & blx or ldr & ldm when loading a pc-relative value) must set bit[0] of the instruction to 1 as this is used to update ESPR.T. If this rule is violated, a INVSTATE exception will be generated. When writing C code, the compiler will take care of this automatically, but this is a common bug which can arise when hand-writing assembly.

This is a bit above my knowledge level :sweat_smile:

I was wondering if maybe we have different compiler versions so the state of the registers is different when that code is attempted to be executed, I tried rebuilding with set(PICO_DEOPTIMIZED_DEBUG ON) so that -O0 is used, but still, same result.

Mr-Bossman commented 2 weeks ago

Oh, I should have caught this earlier since your gdb was armed, but I assumed it was multiarch or something. Anyway, this only works on the riscv core right now, though it should be trival to get it running on arm. Can you recompile using the riscv core.

arturo182 commented 2 weeks ago

One detail I noticed (probably not related, but worth noting) is that in your screenshot the partition is 4M, but in the checked in json, it's 8M: https://github.com/Mr-Bossman/pi-pico2-linux/blob/master/psram-bootloader/partition_table.json#L18

When I tried and changed ti to 4M, flashing the kernel fails because it is too large for 4M.

> picotool load -fxp 0 ../buildroot/output/images/flash-image.bin
Downloading into partition 0:
  00010000->00410000
ERROR: File size 0x600000 is too big to fit in partition size 0x400000
Mr-Bossman commented 2 weeks ago

The partition size is unrelated. It was just because I didn't have a root fs in that screenshot

arturo182 commented 2 weeks ago

Oh, I should have caught this earlier since your gdb was armed, but I assumed it was multiarch or something. Anyway, this only works on the riscv core right now, though it should be trival to get it running on arm. Can you recompile using the riscv core.

Oh this would explain a lot, it's because I am compiling from command line and this whole block was making a lot of assumptions so I deleted it, but I didn't realize that it was also setting the PLATFORM to riscv! https://github.com/Mr-Bossman/pi-pico2-linux/blob/master/psram-bootloader/CMakeLists.txt#L12

Building the toolchain now, will report back, but it seems likely this is the cause.

Sorry for my oversight!

arturo182 commented 2 weeks ago

Yes, that was the issue, do I feel silly 😅 Sorry for wasting your time, and on your vacation!

Now I get the kernel booting to a point but erroring eventually 🤔 

PSRAM ID: 5d 52
PSRAM setup complete. PSRAM size 0x800000 (8388608)
Jumping to 0x117ffffc, aligned from 0x117ffffc
Function pointers: 0x82 0x80
Partition Start 0x10010000, End 0x810000, Size: 0x800000

Rom dump:
10010000   d0 0d fe ed 00 00 04 8f  00 00 00 38 00 00 03 cc  |...........8....|
10010010   00 00 00 28 00 00 00 11  00 00 00 10 00 00 00 00  |...(............|
00000030
Kernel offset: 0x0000048f

Ram dump:
11000000   6f 00 c0 05 00 00 00 00  00 00 00 00 00 00 00 00  |o...............|
11000010   c8 59 32 00 00 00 00 00  00 00 00 00 00 00 00 00  |.Y2.............|
00000030

Jumping to kernel at 0x11000000 and DT at 0x10010000
If you are using USB serial, please connect over the hardware serial port.
[    0.000000] Linux version 6.10.0 (arturo182@arturo182-l480) (riscv32-buildroot-linux-uclibc-gcc.br_real (Buildroot -g32161a3c) 13.3.0, GNU ld (GNU Binutils) 
2.41) #1 Tue Aug 27 19:28:18 CEST 2024
[    0.000000] Machine model: riscv-minimal-nommu,qemu
[    0.000000] earlycon: pl11 at MMIO32 0x40070000 (options '')
[    0.000000] printk: legacy bootconsole [pl11] enabled
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000011000000-0x00000000117fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000011000000-0x00000000117fffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000011000000-0x00000000117fffff]
[    0.000000] riscv: base ISA extensions aim
[    0.000000] riscv: ELF capabilities aim
[    0.000000] Kernel command line: earlycon=pl011,mmio32,0x40070000 console=ttyAMA0 root=/dev/mtdblock0 ro rootwait
[    0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Built 1 zonelists, mobility grouping off.  Total pages: 2048
[    0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off
[    0.000000] Memory: 4856K/8192K available (2424K kernel code, 310K rwdata, 247K rodata, 131K init, 102K bss, 3336K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-1, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 32 local interrupts mapped
[    0.000000] timer_probe: no matching timers found
[    0.000000] Console: colour dummy device 80x25
[    0.000000] sched_clock: 32 bits at 250 Hz, resolution 4000000ns, wraps every 8589934590000000ns
[    0.000000] Calibrating delay loop (skipped), value calculated using timer frequency.. 2.00 BogoMIPS (lpj=4000)
[    0.000000] pid_max: default: 4096 minimum: 301
[    0.000000] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.000000] devtmpfs: initialized
[    0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.000000] futex hash table entries: 16 (order: -5, 192 bytes, linear)
[    0.000000] Serial: AMBA PL011 UART driver
[    0.000000] workingset: timestamp_bits=30 max_order=11 bucket_order=0
[    0.000000] Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled
[    0.000000] physmap-flash physmap-flash.0: physmap platform flash device: [mem 0x10410000-0x1060ffff]
[    0.000000] clk: Disabling unused clocks
[    0.000000] PM: genpd: Disabling unused power domains
[    0.000000] Warning: unable to open an initial console.
[    0.000000] EXT4-fs (mtdblock0): write access unavailable, skipping orphan cleanup
[    0.000000] EXT4-fs (mtdblock0): mounted filesystem eba88cbe-bd03-467e-8ccd-4c5bc9d70ecc ro with ordered data mode. Quota mode: disabled.
[    0.000000] VFS: Mounted root (ext4 filesystem) readonly on device 31:0.
[    0.000000] devtmpfs: mounted
[    0.000000] Freeing unused kernel image (initmem) memory: 128K
[    0.000000] This architecture does not have kernel memory protection.
[    0.000000] Run /sbin/init as init process
[    0.000000] mount: page allocation failure: order:8, mode:0xcc0(GFP_KERNEL), nodemask=(null)
[    0.000000] CPU: 0 PID: 18 Comm: mount Not tainted 6.10.0 #1
[    0.000000] Hardware name: riscv-minimal-nommu,qemu (DT)
[    0.000000] Call Trace:
[    0.000000] [<11003fd8>] 0x11003fd8
[    0.000000] [<112556dc>] 0x112556dc
[    0.000000] [<11259728>] 0x11259728
[    0.000000] [<1125976c>] 0x1125976c
[    0.000000] [<1108792c>] 0x1108792c
[    0.000000] [<11087ee4>] 0x11087ee4
[    0.000000] [<110881a8>] 0x110881a8
[    0.000000] [<110883cc>] 0x110883cc
[    0.000000] [<110884dc>] 0x110884dc
[    0.000000] [<11082c70>] 0x11082c70
[    0.000000] [<11079bc4>] 0x11079bc4
[    0.000000] [<11079c50>] 0x11079c50
[    0.000000] [<110d688c>] 0x110d688c
[    0.000000] [<110978a0>] 0x110978a0
[    0.000000] [<11098094>] 0x11098094
[    0.000000] [<11098988>] 0x11098988
[    0.000000] [<110989cc>] 0x110989cc
[    0.000000] [<1125a004>] 0x1125a004
[    0.000000] [<1125f0f0>] 0x1125f0f0
[    0.000000] Mem-Info:
[    0.000000] active_anon:0 inactive_anon:0 isolated_anon:0
[    0.000000]  active_file:61 inactive_file:45 isolated_file:0
[    0.000000]  unevictable:0 dirty:0 writeback:0
[    0.000000]  slab_reclaimable:0 slab_unreclaimable:195
[    0.000000]  mapped:0 shmem:0 pagetables:0
[    0.000000]  sec_pagetables:0 bounce:0
[    0.000000]  kernel_misc_reclaimable:0
[    0.000000]  free:760 free_pcp:0 free_cma:0
[    0.000000] Node 0 active_anon:0kB inactive_anon:0kB active_file:244kB inactive_file:180kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB d
irty:0kB writeback:0kB shmem:0kB writeback_tmp:0kB kernel_stack:136kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
[    0.000000] Normal free:3040kB boost:0kB min:276kB low:344kB high:412kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:244kB inactive_
file:180kB unevictable:0kB writepending:0kB present:8192kB managed:4984kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[    0.000000] lowmem_reserve[]: 0 0
[    0.000000] Normal: 8*4kB (U) 4*8kB (UM) 2*16kB (U) 2*32kB (UM) 5*64kB (UM) 2*128kB (UM) 3*256kB (UM) 3*512kB (UM) 0*1024kB 0*2048kB 0*4096kB = 3040kB
[    0.000000] 106 total pagecache pages
[    0.000000] 2048 pages RAM
[    0.000000] 0 pages HighMem/MovableOnly
[    0.000000] 802 pages reserved
[    0.000000] nommu: Allocation of length 577536 from process 18 (mount) failed
[    0.000000] Mem-Info:
[    0.000000] active_anon:0 inactive_anon:0 isolated_anon:0
[    0.000000]  active_file:61 inactive_file:45 isolated_file:0
[    0.000000]  unevictable:0 dirty:0 writeback:0
[    0.000000]  slab_reclaimable:0 slab_unreclaimable:195
[    0.000000]  mapped:0 shmem:0 pagetables:0
[    0.000000]  sec_pagetables:0 bounce:0
[    0.000000]  kernel_misc_reclaimable:0
[    0.000000]  free:760 free_pcp:0 free_cma:0
[    0.000000] Node 0 active_anon:0kB inactive_anon:0kB active_file:244kB inactive_file:180kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB d
irty:0kB writeback:0kB shmem:0kB writeback_tmp:0kB kernel_stack:136kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
[    0.000000] Normal free:3040kB boost:0kB min:276kB low:344kB high:412kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:244kB inactive_
file:180kB unevictable:0kB writepending:0kB present:8192kB managed:4984kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[    0.000000] lowmem_reserve[]: 0 0
[    0.000000] Normal: 8*4kB (U) 4*8kB (UM) 2*16kB (U) 2*32kB (UM) 5*64kB (UM) 2*128kB (UM) 3*256kB (UM) 3*512kB (UM) 0*1024kB 0*2048kB 0*4096kB = 3040kB
[    0.000000] 106 total pagecache pages
[    0.000000] 2048 pages RAM
[    0.000000] 0 pages HighMem/MovableOnly
[    0.000000] 802 pages reserved
[    0.000000] binfmt_flat: Unable to allocate RAM for process text/data, errno -12

With -12 being ENOMEM

Mr-Bossman commented 2 weeks ago

Yes, it runs out of memory, unfortunately. I haven't gotten a shell working.... you can get the kernel to fit in ~2.3Mb by disabling a bunch of stuff in the config. Then it starts sh, but then the shell runs out of memory, too. I should add a helloworld init.

Why didn't you use the build process in the Readme and makefile?

arturo182 commented 2 weeks ago

Well, I did use it, it's just that your cmake file expects the sdk and toolchain to be in a specific location and I don't use the vscode extension so I have them checked out in another place.

But great news on the Linux front then, if you run into the same issue then it's not my fault this time :D

Mr-Bossman commented 2 weeks ago

Lol I just used the extension to install the SDK to a standard location hopeing for this exact thing not to happen. Anyway it's fine. Would be nice to know how to change the sdk and tools location though.

arturo182 commented 2 weeks ago

The cmake file should probably check if PICO_SDK_PATH is set in the env and only set it to the default location if not already defined.

I've been trying to disable as many options and lower as many values in the kernel as possible, but no shell so far 😅

Mr-Bossman commented 2 weeks ago

The cmake file should probably check if PICO_SDK_PATH is set in the env and only set it to the default location if not already defined.

It seems like the vscode extension is a bit broken and overwrites them. i updated it and it seems to be somewhat fixed.

I've been trying to disable as many options and lower as many values in the kernel as possible, but no shell so far 😅

disdi commented 1 week ago

Yes, it runs out of memory, unfortunately. I haven't gotten a shell working.... you can get the kernel to fit in ~2.3Mb by disabling a bunch of stuff in the config.

I cannot get the Kernel Image to reduce below 4Mb. Can you share the config file which can fit it in 2.3 Mb ?

Mr-Bossman commented 1 week ago

@disdi if you set the init system to none it will start /bin/sh and will succeed, but for some reason panics. https://github.com/Mr-Bossman/pi-pico2-linux/tree/test-init

Please create a new issue next time.