search

polarfire-soc / polarfire-soc-bare-metal-examples

Bare metal example software projects for PolarFire SoC
25 stars 14 forks source link

How to make work the pmp example? #5

Closed fcuzzocrea closed 2 years ago

fcuzzocrea commented 2 years ago

This issue can be seen as a follow up of what I tried in here https://github.com/riscv/riscv-openocd/issues/662 in order to be able to program an arbitrary custom application into the LIM using OpenOCD, when running on my custom bootloader (both the bootloader and the application are built using the baremetal library, the bootloader is built using IMAGE_LOADED_BY_BOOTLOADER 0 and the application is built using IMAGE_LOADED_BY_BOOTLOADER 1) in my custom build environment (not SoftConsole based). Not being able to achieve my goal, I decided to analyze how this example was working, but I wasn't able to make it work.

To resume what I did:

Actually I also tried single stepping but it is not clear the cause why I end up in trap_from_machine_mode (at least to me):

(gdb) thread apply all set $pc = _start

Thread 1 (Remote target):
(gdb) bt
#0  reset_vector () at ../src/platform/mpfs_hal/startup_gcc/mss_entry.S:300
(gdb) si
0x0000000080000004  300     la a4, trap_vector
(gdb) si
301     csrw mtvec, a4          # initalise machine trap vector address
(gdb) si
304     csrr    a5, mtvec
(gdb) si
305     bne a4, a5, 2b
(gdb) si
311     beqz a0, 3f
(gdb) si
312     csrw mideleg, 0
(gdb) si
313     csrw medeleg, 0
(gdb) si
316     csrw mscratch, zero
(gdb) si
317     csrw mcause, zero
(gdb) si
318     csrw mepc, zero
(gdb) si
323     beqz a0, 1f
(gdb) si
325     fscsr x0
(gdb) si
trap_vector () at ../src/platform/mpfs_hal/startup_gcc/mss_entry.S:391
391     addi sp, sp, -INTEGER_CONTEXT_SIZE     # moves sp down stack to make I
(gdb) si
394     STORE sp, 2*REGBYTES(sp)               # sp
(gdb) si
395     STORE a0, 10*REGBYTES(sp)              # save a0,a1 in the created CONTEXT
(gdb) si
396     STORE a1, 11*REGBYTES(sp)
(gdb) si
397     STORE ra, 1*REGBYTES(sp)
(gdb) si
398     STORE gp, 3*REGBYTES(sp)
(gdb) si
399     STORE tp, 4*REGBYTES(sp)
(gdb) si
400     STORE t0, 5*REGBYTES(sp)
(gdb) si
401     STORE t1, 6*REGBYTES(sp)
(gdb) si
402     STORE t2, 7*REGBYTES(sp)
(gdb) si
403     STORE s0, 8*REGBYTES(sp)
(gdb) si
404     STORE s1, 9*REGBYTES(sp)
(gdb) si
405     STORE a2,12*REGBYTES(sp)
(gdb) si
406     STORE a3,13*REGBYTES(sp)
(gdb) si
407     STORE a4,14*REGBYTES(sp)
(gdb) si
408     STORE a5,15*REGBYTES(sp)
(gdb) si
409     STORE a6,16*REGBYTES(sp)
(gdb) si
410     STORE a7,17*REGBYTES(sp)
(gdb) si
411     STORE s2,18*REGBYTES(sp)
(gdb) si
412     STORE s3,19*REGBYTES(sp)
(gdb) si
413     STORE s4,20*REGBYTES(sp)
(gdb) si
414     STORE s5,21*REGBYTES(sp)
(gdb) si
415     STORE s6,22*REGBYTES(sp)
(gdb) si
416     STORE s7,23*REGBYTES(sp)
(gdb) si
417     STORE s8,24*REGBYTES(sp)
(gdb) si
418     STORE s9,25*REGBYTES(sp)
(gdb) si
419     STORE s10,26*REGBYTES(sp)
(gdb) si
420     STORE s11,27*REGBYTES(sp)
(gdb) si
421     STORE t3,28*REGBYTES(sp)
(gdb) si
422     STORE t4,29*REGBYTES(sp)
(gdb) si
423     STORE t5,30*REGBYTES(sp)
(gdb) si
424     STORE t6,31*REGBYTES(sp)
(gdb) si
426     mv a0, sp                          # a0 <- regs
(gdb) si
432     csrr a1, mbadaddr                 # useful for anaysis when things go wrong
(gdb) si
433     csrr a2, mepc
(gdb) si
434     jal trap_from_machine_mode
(gdb) si
trap_from_machine_mode (regs=0x8041258, dummy=3149939, mepc=2147483696) at ../src/platform/mpfs_hal/common/mss_mtrap.c:760
760     volatile uintptr_t mcause = read_csr(mcause);
(gdb) si
0x0000000080000a6a  760     volatile uintptr_t mcause = read_csr(mcause);
(gdb) si
0x0000000080000a6c  760     volatile uintptr_t mcause = read_csr(mcause);
(gdb) si
0x0000000080000a6e  760     volatile uintptr_t mcause = read_csr(mcause);
(gdb) si
0x0000000080000a70  760     volatile uintptr_t mcause = read_csr(mcause);
(gdb) si
0x0000000080000a72  760     volatile uintptr_t mcause = read_csr(mcause);
(gdb) si
760     volatile uintptr_t mcause = read_csr(mcause);
(gdb) si
762     if (((mcause & MCAUSE_INT) == MCAUSE_INT) && ((mcause & MCAUSE_CAUSE)  > 15U)&& ((mcause & MCAUSE_CAUSE)  < 64U))
(gdb) si
0x0000000080000a7a  762     if (((mcause & MCAUSE_INT) == MCAUSE_INT) && ((mcause & MCAUSE_CAUSE)  > 15U)&& ((mcause & MCAUSE_CAUSE)  < 64U))
(gdb) si
766     else if (((mcause & MCAUSE_INT) == MCAUSE_INT) && ((mcause & MCAUSE_CAUSE)  == IRQ_M_EXT))
(gdb) si
0x0000000080000aaa  766     else if (((mcause & MCAUSE_INT) == MCAUSE_INT) && ((mcause & MCAUSE_CAUSE)  == IRQ_M_EXT))
(gdb) si
770     else if (((mcause & MCAUSE_INT) == MCAUSE_INT) && ((mcause & MCAUSE_CAUSE)  == IRQ_M_SOFT))
(gdb) si
0x0000000080000ac2  770     else if (((mcause & MCAUSE_INT) == MCAUSE_INT) && ((mcause & MCAUSE_CAUSE)  == IRQ_M_SOFT))
(gdb) si
774     else if (((mcause & MCAUSE_INT) == MCAUSE_INT) && ((mcause & MCAUSE_CAUSE)  == IRQ_M_TIMER))
(gdb) si
0x0000000080000ada  774     else if (((mcause & MCAUSE_INT) == MCAUSE_INT) && ((mcause & MCAUSE_CAUSE)  == IRQ_M_TIMER))
(gdb) si
778     else if ((mcause == CAUSE_STORE_ACCESS) | (mcause == CAUSE_LOAD_ACCESS) | (mcause == CAUSE_FETCH_ACCESS))
(gdb) si
0x0000000080000af2  778     else if ((mcause == CAUSE_STORE_ACCESS) | (mcause == CAUSE_LOAD_ACCESS) | (mcause == CAUSE_FETCH_ACCESS))
(gdb) si
0x0000000080000af4  778     else if ((mcause == CAUSE_STORE_ACCESS) | (mcause == CAUSE_LOAD_ACCESS) | (mcause == CAUSE_FETCH_ACCESS))
(gdb) si
0x0000000080000af6  778     else if ((mcause == CAUSE_STORE_ACCESS) | (mcause == CAUSE_LOAD_ACCESS) | (mcause == CAUSE_FETCH_ACCESS))
(gdb) si
0x0000000080000af8  778     else if ((mcause == CAUSE_STORE_ACCESS) | (mcause == CAUSE_LOAD_ACCESS) | (mcause == CAUSE_FETCH_ACCESS))
(gdb) si
0x0000000080000afa  778     else if ((mcause == CAUSE_STORE_ACCESS) | (mcause == CAUSE_LOAD_ACCESS) | (mcause == CAUSE_FETCH_ACCESS))
(gdb) si
0x0000000080000afe  778     else if ((mcause == CAUSE_STORE_ACCESS) | (mcause == CAUSE_LOAD_ACCESS) | (mcause == CAUSE_FETCH_ACCESS))
(gdb) si
0x0000000080000b02  778     else if ((mcause == CAUSE_STORE_ACCESS) | (mcause == CAUSE_LOAD_ACCESS) | (mcause == CAUSE_FETCH_ACCESS))
(gdb) si
0x0000000080000b04  778     else if ((mcause == CAUSE_STORE_ACCESS) | (mcause == CAUSE_LOAD_ACCESS) | (mcause == CAUSE_FETCH_ACCESS))
(gdb) si
0x0000000080000b06  778     else if ((mcause == CAUSE_STORE_ACCESS) | (mcause == CAUSE_LOAD_ACCESS) | (mcause == CAUSE_FETCH_ACCESS))
(gdb) si
0x0000000080000b0a  778     else if ((mcause == CAUSE_STORE_ACCESS) | (mcause == CAUSE_LOAD_ACCESS) | (mcause == CAUSE_FETCH_ACCESS))
(gdb) si
^[[A0x0000000080000b0c  778     else if ((mcause == CAUSE_STORE_ACCESS) | (mcause == CAUSE_LOAD_ACCESS) | (mcause == CAUSE_FETCH_ACCESS))
(gdb) si
^[[A805             i++;        /* added some code as SC debugger hangs if in loop doing nothing */
(gdb) si
^[[A806             if(i == 0x1000U)
(gdb) si
805             i++;        /* added some code as SC debugger hangs if in loop doing nothing */

JFYI, I also tried to load the application using SoftConsole and the mpfs-pmp-app-u54-1 hw hart1 attach configuration, but nothing happens.

hughbreslin commented 2 years ago

Hi @fcuzzocrea could you try setting your board to boot mode 0 (easiest way is the SoftConsole external tool) and try again? I'm wondering if Linux came up in the background from your eMMC when the HSS started up?

fcuzzocrea commented 2 years ago

Yes, I can try that. For what I can say, before trying to flash the example, I manually stopped Linux from autoboot by hitting a key.

fcuzzocrea commented 2 years ago

@hughbreslin but I have to recompile the whole HSS ? I just tried trough softconsole external tools to set Bootmode 0, and the procedure ended up correctly according to the logs, but now when I switch off and on the board the HSS splash screen does not come up anymore

hughbreslin commented 2 years ago

The HSS is still present just that boot mode 0 is idle boot - you could re-program with the reference design job which should re-set the boot mode or re-build the HSS and either use the boot mode programmer in softconsole or use make program from the command line to program the eNVM and set the boot mode

fcuzzocrea commented 2 years ago

Alright - then, selecting idle boot using SoftConsole had a worse effect, at least on my machine. I am not even able to load the program on the board. What I did after selecting the idle boot from soft console was:

For help, type "help". Type "apropos word" to search for commands related to "word"... Really redefine built-in command "remote"? (y or n) [answered Y; input not from terminal] Reading symbols from mpfs-pmp-app-u54-1.elf... (gdb) set mem inaccessible-by-default off (gdb) set $target_riscv=1 (gdb) set arch riscv:rv64 The target architecture is assumed to be riscv:rv64 (gdb) target remote localhost:3333 0x000000002000312e in ?? () Loading section .text, size 0x2780 lma 0x80000000 Loading section .sdata, size 0x10 lma 0x80002780 Ignoring packet error, continuing... Loading section .data, size 0xe30 lma 0x80002790 Load failed

 - Observe the following output from OpenOCD:

Info : accepting 'gdb' connection on tcp/3333 Error: Timed out after 2s waiting for busy to go low (abstractcs=0x10001002). Increase the timeout with riscv set_command_timeout_sec. Error: Abstract command ended in error 'busy' (abstractcs=0x10001102) Error: Timed out after 2s waiting for busy to go low (abstractcs=0x10001102). Increase the timeout with riscv set_command_timeout_sec. Error: Abstract command ended in error 'busy' (abstractcs=0x10001102) Error: Timed out after 2s waiting for busy to go low (abstractcs=0x10001102). Increase the timeout with riscv set_command_timeout_sec. Error: Abstract command ended in error 'busy' (abstractcs=0x10001102) Error: Timed out after 2s waiting for busy to go low (abstractcs=0x10001102). Increase the timeout with riscv set_command_timeout_sec. Error: Failed to read priv register. Error: Abstract command ended in error 'busy' (abstractcs=0x10001102) Error: Timed out after 2s waiting for busy to go low (abstractcs=0x10001102). Increase the timeout with riscv set_command_timeout_sec. Warn : negative acknowledgment, but no packet pending Warn : negative acknowledgment, but no packet pending Warn : negative acknowledgment, but no packet pending Error: Abstract command ended in error 'busy' (abstractcs=0x10001102) Error: Timed out after 2s waiting for busy to go low (abstractcs=0x10001102). Increase the timeout with riscv set_command_timeout_sec. Error: Failed to read priv register. Error: Abstract command ended in error 'busy' (abstractcs=0x10001102) Error: Timed out after 2s waiting for busy to go low (abstractcs=0x10001102). Increase the timeout with riscv set_command_timeout_sec. Warn : negative acknowledgment, but no packet pending Warn : keep_alive() was not invoked in the 1000ms timelimit. GDB alive packet not sent! (7002). Workaround: increase "set remotetimeout" in GDB ^Cshutdown command invoked Error: Abstract command ended in error 'busy' (abstractcs=0x10001102) Error: Timed out after 2s waiting for busy to go low (abstractcs=0x10001102). Increase the timeout with riscv set_command_timeout_sec. Error: Abstract command ended in error 'busy' (abstractcs=0x10001102) Error: Timed out after 2s waiting for busy to go low (abstractcs=0x10001102). Increase the timeout with riscv set_command_timeout_sec. Error: unable to resume hart 1 Error: dmstatus =0x00000382 Error executing event gdb-detach on target mpfs.hart1_u54_1:

Info : Embedded FlashPro6 (revision B): closing the device


I had to manually kill it
hughbreslin commented 2 years ago

Apologies I misread your first message - did you program the demo into emmc / SD card after building it? There are additional steps in the readme to run this demo as a HSS payload.

fcuzzocrea commented 2 years ago

Aaah, sorry, I was too fast in trying it.

So in essence this application runs in the same way of my custom application run. In my use case, instead of generating an HSS payload I generate a payload for my custom bootloader which then load the app (which has been programmed into an SPI flash), copies it into the LIM and jump to it using the jump_to_application() funtcion. And this chain works correctly.

What I want to do is to be able to load the raw application (so to speak, the ELF file produced after the build), directly into the LIM, using OpenOCD or an external JTAG programmer (for example, a Lauterbach debugger).

This thing works when the application is compiled with IMAGE_LOADED_BY_BOOTLOADER 1, as all the code which is needed to initialize the SoC is executed, but it does not in the following scenario:

In this scenario you end up into trap_from_machine_mode(), for reasons I fail to understand (as in the case of loading the application as payload and then jumping with jump_to_application() it loads correctly).

The usefulness of this is to be able to attacch GDB to the running application in order to debug the code (either using SoftConsole or directly GDB). Is it possible to achieve this goal using this example? So at least I have a starting working point in SoftConsole environment to understand the mechanism and then replicate it into my custom bootloader/application/build environment.

hughbreslin commented 2 years ago

Hi @fcuzzocrea I'm a bit confused - have you modified the application to run from the LIM? The standard configuration is to run from the DDR.

Is there a typo here by any chance? The usefulness of this is to be able to attacch GDB to the running application in order to debug the code (either using SoftConsole or directly GDB). Is it possible to achieve this goal using this example? - the SoftConsole example contains a debug configuration to attach to the running application, have you tried this when running the standard configuration the demo is set up for when it runs as a payload?

Also is the 0 here a typo? your application is compiled with IMAGE_LOADED_BY_BOOTLOADER 0, so it expects that a bootloader already initialized the SoC, and is linked against the LIM - if set to 0 the application should not expect a bootloader to have initialized the SoC.

fcuzzocrea commented 2 years ago

No, I did not touched the application. And yes that was a typo. And probably I tried the wrong example.

Actually, my goal was not to achieve an attach configuration, my goal was to be able to load into the LIM (or RAM, whatever) an application built with IMAGE_LOADED_BY_BOOTLOADER 1 trough an external JTAG debugger and start executing it (without the need of programming it first into any external medium).

This is useful when you are debugging an application which is supposed to be loaded by a bootloader, and it is linked against he LIM (or equivalently, the RAM) but you want to quickly try it, without programming it into an external medium before.

Actually, today, I was partially able to achieve my goal with this snippet of code (kanged and adapted from the jump_from_application() which I found into the examples), which is run by my bootloader after the SoC has been initialized:

void wait_for_debugger(HLS_DATA* hls, MODE_CHOICE mode_choice,
                       uint64_t next_addr)
{
    /* Store current hardid */
    uint32_t hartid = read_csr(mhartid);

    /* Restore PLIC to known state */
    __disable_irq();
    PLIC_init();

    /* Disable all interrupts: */
    write_csr(mie, 0);

    switch (mode_choice) {
    default:
    case M_MODE:
        /**
         * User application execution should now start and never return
         * here....
         */
        write_csr(mepc, next_addr);
        break;
    case S_MODE:
        /**
         * User application execution should now start and never return
         * here....
         */
        write_csr(mepc, next_addr);
        break;
    }

    register unsigned long a0 asm("a0") = hartid;
    register unsigned long a1 asm("a1") = (unsigned long)hls;

    /* Hold for debugger to upload the app */
    __asm__("ebreak");

    __asm__ __volatile__("mret" : : "r"(a0), "r"(a1));
    __builtin_unreachable();
}

of course, since my application is linked against the LIM, the entry point is 0x8000000, so I just hardcode this value into next_addr.

What happens here is that after i select M mode and store into the register the needfuls, a break point is triggered, so the code halts and waits.

At this point, with Trace32 I am able to execute an attach to the bootloader, manually load my application into the LIM, the program counter is set to 0x8000000, and when I press GO the application loads correctly.

This does not work when I use OpenOCD + GDB though. For some reasons, in this specific scenario the board ends up into a trap.

What I do is:

fcuzzocrea commented 2 years ago

Just a brief update to let you know that I was able to achieve my goal using ebreak. Probably it is just an hack, but maybe could be useful for someone else in future.

Basically I have created a wait_for_debugger() function:

void wait_for_debugger(HLS_DATA* hls, MODE_CHOICE mode_choice,
                       uint64_t next_addr)
{
    /* Store current hardid */
    uint32_t hartid = read_csr(mhartid);

    /* Restore PLIC to known state */
    __disable_irq();
    PLIC_init();

    /* Disable all interrupts: */
    write_csr(mie, 0);

    switch (mode_choice) {
    default:
    case M_MODE:
        /**
         * User application execution should now start and never return
         * here....
         */
        write_csr(mepc, next_addr);
        break;
    case S_MODE:
        /**
         * User application execution should now start and never return
         * here....
         */
        write_csr(mepc, next_addr);
        break;
    }

    register unsigned long a0 asm("a0") = hartid;
    register unsigned long a1 asm("a1") = (unsigned long)hls;

    /* Hold for debugger to upload the app */
    __asm__("ebreak");

    __asm__ __volatile__("mret" : : "r"(a0), "r"(a1));
    __builtin_unreachable();
}

Of course I know the value of next_addr as the entry point of the elf I want to load is the start of the LIM, so I can just hardcode it.

When my bootloader is started from a debugging environment (Trace32 or OpenOCD+GDB), it executes the function and then hits the ebreak, so wait for the application to be loaded in the LIM.

With GDB I do:

fcuzzocrea@Latitude-5420:~$ riscv64-unknown-elf-gdb build/debug/c3app.elf 
GNU gdb (GDB) 10.1
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "--host=x86_64-pc-linux-gnu --target=riscv64-unknown-elf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Really redefine built-in command "remote"? (y or n) [answered Y; input not from terminal]
Reading symbols from build/debug/c3app.elf...
(gdb) set mem inaccessible-by-default off
(gdb) set $target_riscv=1
(gdb) set arch riscv:rv64
The target architecture is set to "riscv:rv64".
(gdb) target extended-remote localhost:3333
Remote debugging using localhost:3333
0x000000002022149a in ?? ()
(gdb) start
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Function "main" not defined.
Make breakpoint pending on future shared library load? (y or [n]) n
Starting program: /home/fcuzzocrea/Documenti/Progetti/core3_template_app/build/debug/c3app.elf 
Disabling abstract command writes to CSRs.

Program received signal SIGTRAP, Trace/breakpoint trap.
0x0000000020221ed8 in ?? ()
(gdb) load
Loading section .text, size 0x22200 lma 0x8000000
Loading section .sdata, size 0x70 lma 0x8022200
Loading section .data, size 0x3930 lma 0x8022270
Loading section .sdram, size 0x1388 lma 0x8025ba0
Start address 0x0000000008000000, load size 159528
Transfer rate: 9 KB/sec, 13294 bytes/write.
(gdb) continue
Continuing.

And my application loads correctly.

As I said, this probably it is just an hack, and can hardly be integrated into an IDE to allow interactive debugging, but at least it is working and I can use GDB from the CLI.

nitindeshpande commented 2 years ago

Since your bootloader is running and probably using the LIM start address for data/bss etc, halting it with ebrake before you load your application to the LIM start address makes sense to me.

Closing this issue since you have a it working with Trace32 as well as OpenOCD+GDB. Please feel free to reopen if we need to discuss this further.