I cannot boot linux. [LITEX-TERM] Got unexpected response from device 'b'E''

nohahanon commented 1 year ago

Hi there!

I'm trying to boot with a Linux image created locally on a tang nano 20k, but I'm encountering an error that says "[LITEX-TERM] Got unexpected response from device 'b'E'" and the log stops without booting. I have attached the detailed log below. Please provide any answer. Thank you!

Files used for booting:
- sipeed_tang_nano_20k.fs: Created with "./make.py --board=sipeed_tang_nano_20k --build"
- sipeed_tang_nano_20k.dtb/dts: Created with "./make.py --board=sipeed_tang_nano_20k --build"
- Image, rootfs.cpio, fw_jump.bin: For the procedure, I followed https://github.com/litex-hub/linux-on-litex-vexriscv#-generating-the-linux-binaries-optional, and I used the ones obtained from https://buildroot.org/ to create each file.
- boot.json: I referred to the boot.json from https://github.com/litex-hub/linux-on-litex-vexriscv/issues/164, and I modified the parts that originally mentioned rv32.dtb and opensbi.bin.
```
{
"Image": "0x40000000",
"sipeed_tang_nano_20k.dtb": "0x40ef0000",
"rootfs.cpio": "0x41000000",
"fw_jump.bin": "0x40f00000"
}
```

log


$ sudo openFPGALoader -f -c ft2232 ~/linux-on-litex-vexriscv/build/sipeed_tang_nano_20k/gateware/sipeed_tang_nano_20k.fs
write to flash
Jtag frequency : requested 6.00MHz   -> real 6.00MHz  
Parse file Parse /home/nohara/linux-on-litex-vexriscv/build/sipeed_tang_nano_20k/gateware/sipeed_tang_nano_20k.fs: 
Done
DONE
Jtag frequency : requested 2.50MHz   -> real 2.00MHz  
Jtag frequency : requested 10.00MHz  -> real 6.00MHz  
erase SRAM Done
Detected: Winbond W25Q64 128 sectors size: 64Mb
Detected: Winbond W25Q64 128 sectors size: 64Mb
RDSR : 00
WIP  : 0
WEL  : 0
BP   : 0
TB   : 0
SRWD : 0
00000000 00000000 00000000 00
Erasing: [==================================================] 100.00%
Done
Writing: [==================================================] 100.00%
Done
$ sudo chmod 666 /dev/ttyUSB*
$ litex_term --serial-boot --images ~/TangNano-20K-example/linux/image5/boot.json /dev/ttyUSB1

litex> reboot

    __   _ __      _  __
   / /  (_) /____ | |/_/
  / /__/ / __/ -_)>  <
 /____/_/\__/\__/_/|_|

Build your hardware, easily!

BIOS CRC passed (5386dd7d)

LiteX git sha1: 36ce71d5

--=============== SoC ==================-- CPU: VexRiscv SMP-LINUX @ 48MHz BUS: WISHBONE 32-bit @ 4GiB CSR: 32-bit data ROM: 64.0KiB SRAM: 6.0KiB L2: 128B SDRAM: 8.0MiB 32-bit @ 48MT/s (CL-2 CWL-2) MAIN-RAM: 8.0MiB

--========== Initialization ============-- Initializing SDRAM @0x40000000... Switching SDRAM to software control. Switching SDRAM to hardware control. Memtest at 0x40000000 (2.0MiB)... Write: 0x40000000-0x40200000 2.0MiB
Read: 0x40000000-0x40200000 2.0MiB
Memtest OK Memspeed at 0x40000000 (Sequential, 2.0MiB)... Write speed: 18.9MiB/s Read speed: 21.9MiB/s

--============== Boot ==================-- Booting from serial... Press Q or ESC to abort boot completely. sL5DdSMmkekro [LITEX-TERM] Received firmware download request from the device. [LITEX-TERM] Uploading /home/nohara/TangNano-20K-example/linux/image5/Image to 0x40000000 (7726264 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Upload complete (9.9KB/s). [LITEX-TERM] Uploading /home/nohara/TangNano-20K-example/linux/image5/sipeed_tang_nano_20k.dtb to 0x40ef0000 (2115 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Got unexpected response from device 'b'E''

nohahanon commented 1 year ago

I attempted to use the SD card containing the above files to boot sdcardboot, but it stopped as follows. Dtb file is not read.

litex> sdcardboot
    Booting from SDCard in SD-Mode...
    Booting from boot.json...
    Copying Image to 0x40000000 (7726264 bytes)...
    [########################################]
    Copying rv32.dtb to 0x40ef0000 (2689 bytes)...
    [  ]

nohahanon commented 1 year ago

I have identified that the processing is halted at the following line in ff.c within the Litex output: if (FfDiskOps->disk_read(fs->pdrv, rbuff, sect, cc) != RES_OK) ABORT(fs, FR_DISK_ERR); I commented out this section, and I was able to successfully move the file to RAM. However, the processing halted at the liftoff stage.

litex> sdcardboot

Booting from SDCard in SD-Mode...
Booting from boot.json...
Copying Image to 0x40000000 (7726264 bytes)...
[########################################]
Copying rv32.dtb to 0x40ef0000 (2689 bytes)...
[########################################]
Copying rootfs.cpio to 0x41000000 (5376512 bytes)...
[########################################]
Copying opensbi.bin to 0x40f00000 (263652 bytes)...
[########################################]
Executing booted program at 0x40f00000

--============= Liftoff! ===============--

avkghost commented 12 months ago

Hi @nohahanon I have the same problem with booting via serial/sdcard boot.

The short answer: It's need to create a customized project for small memory boards.

TL;DR

I've found that the build system generated an incorrect device tree and system configuration. The board has only 8 MB of SDRAM, as shown during self-diagnosis, but the addresses generated in the device tree and the base address for opensbi are incorrect and lie beyond the 8 MB limit. The address 0x40f00000 corresponds to 15 MB. You can try to read addresses 0x4000000 and 0x40f00000 via mem_read and got different results for a first and a second address (second is incorrect address for the device). ### make.py #### part of device tree generated by make.py ``` memory@40000000 { device_type = "memory"; reg = <0x40000000 0x800000>; }; reserved-memory { #address-cells = <1>; #size-cells = <1>; ranges; opensbi@40f00000 { reg = <0x40f00000 0x80000>; }; }; ``` #### boot.json ``` { "Image": "0x40000000", "rv32.dtb": "0x40ef0000", "rootfs.cpio": "0x41000000", "opensbi.bin": "0x40f00000" } ``` ### A tang nano 20k linux [example](https://github.com/sipeed/TangNano-20K-example/tree/main/litex) #### part of device tree from project contains right adresses in memory below 8 MB limit ``` memory@40000000 { device_type = "memory"; reg = <0x40000000 0x800000>; }; reserved-memory { #address-cells = <1>; #size-cells = <1>; ranges; opensbi@407c0000 { reg = <0x407c0000 0x20000>; }; }; ``` #### boot.json ``` { "Image": "0x40000000", "sipeed_tang_nano_20k.dtb": "0x40780000", "opensbi.bin.tangnano20k": "0x407c0000" } ``` The problem is in a [litex/soc/cores/cpu/vexriscv_smp/core.py](https://github.com/enjoy-digital/litex/blob/728afdd7589c262cc5c685052bf1fb16eea9305d/litex/soc/cores/cpu/vexriscv_smp/core.py#L439) ``` soc.bus.add_region("opensbi", SoCRegion(origin=self.mem_map["main_ram"] + 0x00f0_0000, size=0x8_0000, cached=True, linker=True)) ``` My suggestions: * Maybe previous versions litex allow to create a correct device tree for 8 MB boards. * Maybe sipeed created own version of litex/linux-on-litex-vexriscv for the tang nano 20k.

nohahanon commented 12 months ago

Hello @avkghost ! Thank you for the advice! I see that there are errors in the generated device tree. I sent an inquiry email to Sipeed last week asking about how to create the Image and dtb. I'm currently waiting for a response from Sipeed (although my expectations are low).

And, I found a successful example of running Linux on the Tang Primer board. It seems that reducing the size of the Linux image is essential, so I will continue my research. https://www.luffca.com/ja/2022/10/linux-tang-primer/

nohahanon commented 12 months ago

I'm currently trying to modify the BusyBox configuration to see if I can create a smaller Linux image. I've outlined the specific steps below. It seems like I can create a smaller image by removing unnecessary commands, but I'm struggling to get it below 7.7MB. If anyone has any suggestions, I would appreciate it.

(get buildroot, extract and cd there)
$ sudo make BR2_EXTERNAL=../linux-on-litex-vexriscv/buildroot/ litex_vexriscv_defconfig
$ sudo make busybox-menuconfig
(select and delete commands.)
$ sudo make

Dolu1990 commented 12 months ago

Enabline RVC compressed instruction may help reaching it (need to be enable on both litex arguments and linux / busybody. Generaly, it reduce the kernel about 30 %

peiceliu commented 12 months ago

Hello,bro @nohahanon , I have also encountered such a problem. Has your problem been resolved so far?

peiceliu commented 12 months ago

Hello @Dolu1990 , I would like to ask if you have any good solutions for processing halted at the liftoff stage.I am looking forward to your reply .Thanks.

nohahanon commented 12 months ago

Hello, bro @peiceliu ,

I confirmed that by changing the # Ethernet block in linux-on-litex-vexriscv/buildroot/board/litex_vexriscv/linux.config as follows, the size of the Image becomes approximately 5.9MB.(7.7MB->5.9MB)

like this

before ``` # Ethernet CONFIG_NET=y CONFIG_PACKET=y CONFIG_PACKET_DIAG=y CONFIG_INET=y CONFIG_NETDEVICES=y CONFIG_NET_VENDOR_LITEX=y CONFIG_LITEX_LITEETH=y ``` after ``` # Ethernet CONFIG_NET=n CONFIG_PACKET=n CONFIG_PACKET_DIAG=n CONFIG_INET=n CONFIG_NETDEVICES=n CONFIG_NET_VENDOR_LITEX=n CONFIG_LITEX_LITEETH=n ```

I am unable to remove unnecessary commands from busybox. Even after executing 'make busybox-menuconfig' and removing some commands, when I build it, the resulting Image is still the same size. Additionally, specifying 'buildroot-minimal.config' in the .config file located directly under the buildroot directory, obtained by extraction, did not change the size of the Image when compared to specifying 'buildroot.config'.

peiceliu commented 12 months ago

Thanks for your advice @nohahanon,after i change y to n,i still can not boot linux successfully. Should i change other files?Such as [litex/soc/cores/cpu/vexriscv_smp/core.py]. If you could give me appropriate advice to make linux to boot successfully on my fpga board, I would be very grateful.I am looking forward to your reply，bro.

avkghost commented 11 months ago

Hello @nohahanon,

I've seen the link you provided, thanks! However, I had neither found a solution for the Linux kernel configuration, nor a solution for litex for a board configuration. So, the example contains information about Tang Primer which has a different FPGA (Anlogic instead of Gowin). In addition, the example shows that an author can create a multicore VexRiscv bitstream and program the FPGA with it.

Kernel configuration - I've tried to get the kernel configuration from itself, but unfortunately, the kernel didn't have it to reduce its size.
SoC configuration - I've also tried to generate a dual-core soc for Tang Nano 20k, but unfortunately, all my configurations did not fill in 20K LUT. The minimal amount of LUT's in for the dual-core that I've got was 51272, but the GW2AR has only 20k. When I used the SoC generator from litex-boards it generated a configuration with more LUTs (~60+k).

As I investigated the readme from TangNano-20K-example I've found that the Linux kernel, device tree, and boot loader provided by Icenowy. I'm going to ask her for details about the kernel configuration.

So, I'm going to create a debug version of a VexRiscv to debug the booting process. First of all, it would help me to understand a configuration difference between my own and @Icenowy 's.

Update:

nohahanon commented 11 months ago

Thanks @avkghost !! I'm looking forward to it!

Icenowy commented 11 months ago

@avkghost The LiteX port is by me, but booting Linux is done by another person, Jisheng Zhang .

avkghost commented 11 months ago

Hi @Icenowy.

Thanks for joining our discussion.
There are my experiments with litex-boards and linux-on-litex-vexriscv.

litex-boards

python -m litex_boards.targets.sipeed_tang_nano_20k --cpu-type vexriscv --cpu-variant linux --csr-csv csr.csv --uart-baudrate 3000000 --build

It produces csr.csv

``` csr_base,buttons,0xf0000000,, csr_base,ctrl,0xf0000800,, csr_base,identifier_mem,0xf0001000,, csr_base,leds,0xf0001800,, csr_base,sdram,0xf0002000,, csr_base,timer0,0xf0002800,, csr_base,uart,0xf0003000,, csr_register,buttons_in,0xf0000000,1,ro csr_register,ctrl_reset,0xf0000800,1,rw csr_register,ctrl_scratch,0xf0000804,1,rw csr_register,ctrl_bus_errors,0xf0000808,1,ro csr_register,leds_out,0xf0001800,1,rw csr_register,sdram_dfii_control,0xf0002000,1,rw csr_register,sdram_dfii_pi0_command,0xf0002004,1,rw csr_register,sdram_dfii_pi0_command_issue,0xf0002008,1,rw csr_register,sdram_dfii_pi0_address,0xf000200c,1,rw csr_register,sdram_dfii_pi0_baddress,0xf0002010,1,rw csr_register,sdram_dfii_pi0_wrdata,0xf0002014,1,rw csr_register,sdram_dfii_pi0_rddata,0xf0002018,1,ro csr_register,timer0_load,0xf0002800,1,rw csr_register,timer0_reload,0xf0002804,1,rw csr_register,timer0_en,0xf0002808,1,rw csr_register,timer0_update_value,0xf000280c,1,rw csr_register,timer0_value,0xf0002810,1,ro csr_register,timer0_ev_status,0xf0002814,1,ro csr_register,timer0_ev_pending,0xf0002818,1,rw csr_register,timer0_ev_enable,0xf000281c,1,rw csr_register,uart_rxtx,0xf0003000,1,rw csr_register,uart_txfull,0xf0003004,1,ro csr_register,uart_rxempty,0xf0003008,1,ro csr_register,uart_ev_status,0xf000300c,1,ro csr_register,uart_ev_pending,0xf0003010,1,rw csr_register,uart_ev_enable,0xf0003014,1,rw csr_register,uart_txempty,0xf0003018,1,ro csr_register,uart_rxfull,0xf000301c,1,ro constant,config_clock_frequency,48000000,, constant,config_cpu_has_interrupt,None,, constant,config_cpu_reset_addr,0,, constant,config_cpu_has_dcache,None,, constant,config_cpu_has_icache,None,, constant,config_cpu_type_vexriscv,None,, constant,config_cpu_variant_linux,None,, constant,config_cpu_human_name,vexriscv_linux,, constant,config_cpu_nop,nop,, constant,config_l2_size,128,, constant,config_csr_data_width,32,, constant,config_csr_alignment,32,, constant,config_bus_standard,wishbone,, constant,config_bus_data_width,32,, constant,config_bus_address_width,32,, constant,config_bus_bursting,0,, constant,timer0_interrupt,1,, constant,uart_interrupt,0,, memory_region,rom,0x00000000,131072,cached memory_region,sram,0x10000000,8192,cached memory_region,main_ram,0x40000000,8388608,cached memory_region,csr,0xf0000000,65536,io ```

As I see, the script produces configuration with wrong addresses.

make.py from linux-on-litex-vexriscv

 ./make.py --board sipeed_tang_nano_20k --uart-baudrate 3000000 --build

produces following csr.csv

``` csr_base,buttons,0xf0000000,, csr_base,ctrl,0xf0000800,, csr_base,uart,0xf0001000,, csr_base,timer0,0xf0001800,, csr_base,identifier_mem,0xf0002000,, csr_base,leds,0xf0002800,, csr_base,sdcard_block2mem,0xf0003000,, csr_base,sdcard_core,0xf0003800,, csr_base,sdcard_irq,0xf0004000,, csr_base,sdcard_mem2block,0xf0004800,, csr_base,sdcard_phy,0xf0005000,, csr_base,sdram,0xf0005800,, csr_register,buttons_in,0xf0000000,1,ro csr_register,ctrl_reset,0xf0000800,1,rw csr_register,ctrl_scratch,0xf0000804,1,rw csr_register,ctrl_bus_errors,0xf0000808,1,ro csr_register,uart_rxtx,0xf0001000,1,rw csr_register,uart_txfull,0xf0001004,1,ro csr_register,uart_rxempty,0xf0001008,1,ro csr_register,uart_ev_status,0xf000100c,1,ro csr_register,uart_ev_pending,0xf0001010,1,rw csr_register,uart_ev_enable,0xf0001014,1,rw csr_register,uart_txempty,0xf0001018,1,ro csr_register,uart_rxfull,0xf000101c,1,ro csr_register,timer0_load,0xf0001800,1,rw csr_register,timer0_reload,0xf0001804,1,rw csr_register,timer0_en,0xf0001808,1,rw csr_register,timer0_update_value,0xf000180c,1,rw csr_register,timer0_value,0xf0001810,1,ro csr_register,timer0_ev_status,0xf0001814,1,ro csr_register,timer0_ev_pending,0xf0001818,1,rw csr_register,timer0_ev_enable,0xf000181c,1,rw csr_register,leds_out,0xf0002800,1,rw csr_register,sdcard_block2mem_dma_base,0xf0003000,2,rw csr_register,sdcard_block2mem_dma_length,0xf0003008,1,rw csr_register,sdcard_block2mem_dma_enable,0xf000300c,1,rw csr_register,sdcard_block2mem_dma_done,0xf0003010,1,ro csr_register,sdcard_block2mem_dma_loop,0xf0003014,1,rw csr_register,sdcard_block2mem_dma_offset,0xf0003018,1,ro csr_register,sdcard_core_cmd_argument,0xf0003800,1,rw csr_register,sdcard_core_cmd_command,0xf0003804,1,rw csr_register,sdcard_core_cmd_send,0xf0003808,1,rw csr_register,sdcard_core_cmd_response,0xf000380c,4,ro csr_register,sdcard_core_cmd_event,0xf000381c,1,ro csr_register,sdcard_core_data_event,0xf0003820,1,ro csr_register,sdcard_core_block_length,0xf0003824,1,rw csr_register,sdcard_core_block_count,0xf0003828,1,rw csr_register,sdcard_irq_status,0xf0004000,1,ro csr_register,sdcard_irq_pending,0xf0004004,1,rw csr_register,sdcard_irq_enable,0xf0004008,1,rw csr_register,sdcard_mem2block_dma_base,0xf0004800,2,rw csr_register,sdcard_mem2block_dma_length,0xf0004808,1,rw csr_register,sdcard_mem2block_dma_enable,0xf000480c,1,rw csr_register,sdcard_mem2block_dma_done,0xf0004810,1,ro csr_register,sdcard_mem2block_dma_loop,0xf0004814,1,rw csr_register,sdcard_mem2block_dma_offset,0xf0004818,1,ro csr_register,sdcard_phy_card_detect,0xf0005000,1,ro csr_register,sdcard_phy_clocker_divider,0xf0005004,1,rw csr_register,sdcard_phy_init_initialize,0xf0005008,1,rw csr_register,sdcard_phy_dataw_status,0xf000500c,1,ro csr_register,sdram_dfii_control,0xf0005800,1,rw csr_register,sdram_dfii_pi0_command,0xf0005804,1,rw csr_register,sdram_dfii_pi0_command_issue,0xf0005808,1,rw csr_register,sdram_dfii_pi0_address,0xf000580c,1,rw csr_register,sdram_dfii_pi0_baddress,0xf0005810,1,rw csr_register,sdram_dfii_pi0_wrdata,0xf0005814,1,rw csr_register,sdram_dfii_pi0_rddata,0xf0005818,1,ro constant,config_clock_frequency,48000000,, constant,config_cpu_has_interrupt,None,, constant,config_cpu_reset_addr,0,, constant,config_cpu_count,1,, constant,config_cpu_isa,rv32i2p0_ma,, constant,config_cpu_mmu,sv32,, constant,config_cpu_dcache_size,4096,, constant,config_cpu_dcache_ways,1,, constant,config_cpu_dcache_block_size,64,, constant,config_cpu_icache_size,4096,, constant,config_cpu_icache_ways,1,, constant,config_cpu_icache_block_size,64,, constant,config_cpu_dtlb_size,4,, constant,config_cpu_dtlb_ways,4,, constant,config_cpu_itlb_size,4,, constant,config_cpu_itlb_ways,4,, constant,config_cpu_type_vexriscv_smp,None,, constant,config_cpu_variant_linux,None,, constant,config_cpu_human_name,vexriscv smp-linux,, constant,config_cpu_nop,nop,, constant,config_bios_no_build_time,None,, constant,config_l2_size,128,, constant,config_csr_data_width,32,, constant,config_csr_alignment,32,, constant,config_bus_standard,wishbone,, constant,config_bus_data_width,32,, constant,config_bus_address_width,32,, constant,config_bus_bursting,0,, constant,sdcard_irq_interrupt,3,, constant,timer0_interrupt,2,, constant,uart_interrupt,1,, memory_region,opensbi,0x40f00000,524288,cached+linker memory_region,plic,0xf0c00000,4194304,io memory_region,clint,0xf0010000,65536,io memory_region,rom,0x00000000,65536,cached memory_region,sram,0x10000000,6144,cached memory_region,main_ram,0x40000000,8388608,cached memory_region,csr,0xf0000000,65536,io ```

It looks (almost?) similar to the device tree provided by TangNano-20K-example But when I was trying to boot this an Image with bootloader I got "Liftoff!". As I understand the configuration that I've built is different than your and I need some corrections in a generator flags or maybe create own configuration with corrected addresses.

Could you please answer a few questions? See below.

Do I need to modify a source code for litex-boards and/or litex?
Do I need to use any additional generator flags rather than I provided above?
Could you please provide an example one for litex-board or linux-on-litex-vexriscv that allow to boot linux kernel?

Thanks for the advice.

Dolu1990 commented 11 months ago

peiceliu

I would say running it in a simulation would give full visibility on what the CPU does, that's the way in generaly always go.

Else there is the possibility to use the JTAG debug, and since recently you can even use the official RISC-V debug spec, which support linux kernel debugging aswell :)

avkghost commented 11 months ago

Update:

@nohahanon FYI

I've successfully run linux on a Tang Nano 20k with the kernel provided in a TangNano-20K-example repository.

I use the linux-on-litex-vexriscv with following parameters/flags:

./make.py --board sipeed_tang_nano_20k --with-rvc --uart-baudrate 3000000 --build

There is a boot log

``` __ _ __ _ __ / / (_) /____ | |/_/ / /__/ / __/ -_)> < /____/_/\__/\__/_/|_| Build your hardware, easily! (c) Copyright 2012-2023 Enjoy-Digital (c) Copyright 2007-2015 M-Labs BIOS CRC passed (b7fd5a52) LiteX git sha1: c0d37665 --=============== SoC ==================-- CPU: VexRiscv SMP-LINUX @ 48MHz BUS: WISHBONE 32-bit @ 4GiB CSR: 32-bit data ROM: 64.0KiB SRAM: 6.0KiB L2: 128B SDRAM: 8.0MiB 32-bit @ 48MT/s (CL-2 CWL-2) MAIN-RAM: 8.0MiB --========== Initialization ============-- Initializing SDRAM @0x40000000... Switching SDRAM to software control. Switching SDRAM to hardware control. Memtest at 0x40000000 (2.0MiB)... Write: 0x40000000-0x40200000 2.0MiB Read: 0x40000000-0x40200000 2.0MiB Memtest OK Memspeed at 0x40000000 (Sequential, 2.0MiB)... Write speed: 18.9MiB/s Read speed: 21.2MiB/s --============== Boot ==================-- Booting from serial... Press Q or ESC to abort boot completely. sL5DdSMmkekro [LITEX-TERM] Received firmware download request from the device. [LITEX-TERM] Uploading ./image/Image to 0x40000000 (1818868 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (128.2KB/s). [LITEX-TERM] Uploading ./image/sipeed_tang_nano_20k.dtb to 0x40780000 (2115 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (92.3KB/s). [LITEX-TERM] Uploading ./image/opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (126.3KB/s). [LITEX-TERM] Booting the device. [LITEX-TERM] Done. Executing booted program at 0x407c0000 --============= Liftoff! ===============-- OpenSBI v0.8-2-ga9ce3ad ____ _____ ____ _____ / __ \ / ____| _ \_ _| | | | |_ __ ___ _ __ | (___ | |_) || | | | | | '_ \ / _ \ '_ \ \___ \| _ < | | | |__| | |_) | __/ | | |____) | |_) || |_ \____/| .__/ \___|_| |_|_____/|____/_____| | | |_| Platform Name : LiteX / VexRiscv-SMP Platform Features : timer,mfdeleg Platform HART Count : 8 Boot HART ID : 0 Boot HART ISA : rv32imacs BOOT HART Features : pmp,scounteren,mcounteren,time BOOT HART PMP Count : 16 Firmware Base : 0x407c0000 Firmware Size : 120 KB Runtime SBI Version : 0.2 MIDELEG : 0x00000222 MEDELEG : 0x0000b109 [ 0.000000] Linux version 6.4.0-rc1+ (zjs@CL-A-00022) (riscv64-linux-gnu-gcc (Debian 12.2.0-13) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #10 Tue May 16 09:20:49 CST 2023 [ 0.000000] earlycon: liteuart0 at I/O port 0x0 (options '') [ 0.000000] Malformed early option 'console' [ 0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '') [ 0.000000] printk: bootconsole [liteuart0] enabled [ 0.000000] OF: reserved mem: 0x407c0000..0x407dffff (128 KiB) map non-reusable opensbi@407c0000 [ 0.000000] Zone ranges: [ 0.000000] Normal [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] SBI specification v0.2 detected [ 0.000000] SBI implementation ID=0x1 Version=0x8 [ 0.000000] SBI TIME extension detected [ 0.000000] SBI IPI extension detected [ 0.000000] SBI RFENCE extension detected [ 0.000000] riscv: base ISA extensions aim [ 0.000000] riscv: ELF capabilities aim [ 0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 [ 0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping off. Total pages: 2032 [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.000000] Memory: 6112K/8192K available (1084K kernel code, 82K rwdata, 110K rodata, 495K init, 49K bss, 2080K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-1, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] riscv-intc: 32 local interrupts mapped [ 0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts. [ 0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0] [ 0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns [ 0.000083] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns [ 0.008170] Console: colour dummy device 80x25 [ 0.011115] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=480000) [ 0.014850] pid_max: default: 4096 minimum: 301 [ 0.019099] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.021834] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.092709] ASID allocator using 9 bits (512 entries) [ 0.123244] devtmpfs: initialized [ 0.233157] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns [ 0.235822] futex hash table entries: 16 (order: -5, 192 bytes, linear) [ 0.288091] platform soc: Fixed dependency cycle(s) with /soc/interrupt-controller@f0c00000 [ 0.368089] clocksource: Switched to clocksource riscv_clocksource [ 1.366525] workingset: timestamp_bits=30 max_order=11 bucket_order=0 [ 2.502945] LiteX SoC Controller driver initialized [ 3.099011] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart [ 3.102848] printk: console [liteuart0] enabled [ 3.102848] printk: console [liteuart0] enabled [ 3.105098] printk: bootconsole [liteuart0] disabled [ 3.105098] printk: bootconsole [liteuart0] disabled [ 3.315434] clk: Disabling unused clocks [ 3.319630] Warning: unable to open an initial console. [ 3.361744] Freeing unused kernel image (initmem) memory: 492K [ 3.363448] Kernel memory protection not selected by kernel config. [ 3.365094] Run /init as init process starting pid 18, tty '/dev/console': '/sbin/getty -L -n -l /bin/sh console 0 vt100' / # ```

At the moment I'm going to experiment with a Linux kernel (reduce size and remove unneeded modules/subsystems).

@Icenowy Could Jisheng Zhang help us with a kernel configuration for the board?

peiceliu commented 11 months ago

Hello @avkghost , I tried the suggestions you provided but I was not able to successfully boot Linux on my sipeed_tang_primer_20k . Here is my log.Could you please provide me with more details about the issue?

~/Desktop/linux-on-litex-vexriscv$ ./make.py --board sipeed_tang_primer_20k --with-rvc --uart-baudrate 3000000 --build --load
~/Desktop/linux-on-litex-vexriscv$ litex_term --speed 3000000 --serial-boot --images=images/boot.json /dev/ttyUSB1 NOTE：I had replaced the files in the images folder with the boot.json and other files of tang_primeri_20k example.But my computer does not respond to this command /Desktop/linux-on-litex-vexriscv$ litex_term --speed 3000000 --serial-boot --images=images/boot.json /dev/ttyUSB1 and it stop here. How can i encounter this problem?Please provide any answer. Thank you!

avkghost commented 11 months ago

Hello @peiceliu, Have you tried accessing a litex bootloader console? Is it possible to write any command in the console? If yes, please use serialboot or reboot command.

Unfortunately, now I do not have the ability to test Tang Primer build. The power went out while I was flashing the board. It is currently not possible to write anything to FPGA memory or software SPI flash.

peiceliu commented 11 months ago

Hello @avkghost ,After the second command,I can not boot litex console.In addition,after the first command,my serial tool displayed messy code instead of boot log .

avkghost commented 11 months ago

@peiceliu I've successfully loaded Tang Primer 20k with Linux. So, I restored my board under the windows (using Gowin programmer). The second problem was with a bitstream. I was not able to program it with openFPGALoader. In some cases, the upload was freezing while flashing. When the bitstream uploaded successfully and FPGA was re-booted it did not start a Litex BIOS.

There is a step-by-step solution that I used for my own board, please take a look below.

Create a software and a gateware by following command line. In my variant flashing and loading into SRAM didn't allow programming the bitstream via openFPGALoader.
```
./make.py --board sipeed_tang_primer_20k --with-rvc --uart-baudrate 3000000 --build 
```
1. Transfer all needed gateware files to windows. (sipeed_tang_nano_20k.cst sipeed_tang_nano_20k_mem.init sipeed_tang_nano_20k.sdc sipeed_tang_nano_20k.v sipeed_tang_nano_20k.fs sipeed_tang_nano_20k_rom.init sipeed_tang_nano_20k_sram.init)
2. Create a new project and add these files.
3. Build the project.
4. Flash it. Loading into SRAM is possible, but after turning off the power, SRAM will be cleared.
5. Create a new directory and copy to it following files: Image, boot.json, root.cpio
6. Copy sipeed_tang_primer_20k.dtb to the directory and rename it to rv32.dtb
7. Run litex_term with the flags you provided.
```
litex_term --speed 3000000 --serial-boot --images=images/boot.json /dev/ttyUSB1
```

There is my boot log

``` litex> reboot __ _ __ _ __ / / (_) /____ | |/_/ / /__/ / __/ -_)> < /____/_/\__/\__/_/|_| Build your hardware, easily! (c) Copyright 2012-2023 Enjoy-Digital (c) Copyright 2007-2015 M-Labs BIOS CRC passed (82ac047a) LiteX git sha1: c0d37665 --=============== SoC ==================-- CPU: VexRiscv SMP-LINUX @ 48MHz BUS: WISHBONE 32-bit @ 4GiB CSR: 32-bit data ROM: 64.0KiB SRAM: 6.0KiB L2: 512B SDRAM: 128.0MiB 16-bit @ 192MT/s (CL-6 CWL-5) MAIN-RAM: 128.0MiB --========== Initialization ============-- Initializing SDRAM @0x40000000... Switching SDRAM to software control. Read leveling: m0, b00: |00000000| delays: - m0, b01: |00000000| delays: - m0, b02: |01110000| delays: 02+-01 m0, b03: |00000000| delays: - best: m0, b02 delays: 02+-01 m1, b00: |00000000| delays: - m1, b01: |00000000| delays: - m1, b02: |01110000| delays: 02+-01 m1, b03: |00000000| delays: - best: m1, b02 delays: 02+-01 Switching SDRAM to hardware control. Memtest at 0x40000000 (2.0MiB)... Write: 0x40000000-0x40200000 2.0MiB Read: 0x40000000-0x40200000 2.0MiB Memtest OK Memspeed at 0x40000000 (Sequential, 2.0MiB)... Write speed: 11.7MiB/s Read speed: 15.6MiB/s --============== Boot ==================-- Booting from serial... Press Q or ESC to abort boot completely. sL5DdSMmkekro [LITEX-TERM] Received firmware download request from the device. [LITEX-TERM] Uploading ./images/Image to 0x40000000 (7531468 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (134.9KB/s). [LITEX-TERM] Uploading ./images/rv32.dtb to 0x40ef0000 (2617 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 160.00us, length: 64) [LITEX-TERM] Upload complete (165.4KB/s). [LITEX-TERM] Uploading ./images/rootfs.cpio to 0x41000000 (3781632 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (131.9KB/s). [LITEX-TERM] Uploading ./images/opensbi.bin to 0x40f00000 (53640 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (121.4KB/s). [LITEX-TERM] Booting the device. [LITEX-TERM] Done. Executing booted program at 0x40f00000 --============= Liftoff! ===============-- OpenSBI v0.8-1-gecf7701 ____ _____ ____ _____ / __ \ / ____| _ \_ _| | | | |_ __ ___ _ __ | (___ | |_) || | | | | | '_ \ / _ \ '_ \ \___ \| _ < | | | |__| | |_) | __/ | | |____) | |_) || |_ \____/| .__/ \___|_| |_|_____/|____/_____| | | |_| Platform Name : LiteX / VexRiscv-SMP Platform Features : timer,mfdeleg Platform HART Count : 8 Boot HART ID : 0 Boot HART ISA : rv32imacs BOOT HART Features : pmp,scounteren,mcounteren,time BOOT HART PMP Count : 16 Firmware Base : 0x40f00000 Firmware Size : 124 KB Runtime SBI Version : 0.2 MIDELEG : 0x00000222 MEDELEG : 0x0000b109 [ 0.000000] Linux version 5.14.0 (florent@panda) (riscv32-buildroot-linux-gnu-gcc.br_real (Buildroot 2021.08-381-g279167ee8d) 10.3.0, GNU ld (GNU Binutils) 2.36.1) #1 SMP Tue Sep 21 12:57:31 CEST 2021 [ 0.000000] earlycon: liteuart0 at I/O port 0x0 (options '') [ 0.000000] Malformed early option 'console' [ 0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '') [ 0.000000] printk: bootconsole [liteuart0] enabled [ 0.000000] Zone ranges: [ 0.000000] Normal [mem 0x0000000040000000-0x0000000047ffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000040000000-0x0000000047ffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x0000000047ffffff] [ 0.000000] SBI specification v0.2 detected [ 0.000000] SBI implementation ID=0x1 Version=0x8 [ 0.000000] SBI TIME extension detected [ 0.000000] SBI IPI extension detected [ 0.000000] SBI RFENCE extension detected [ 0.000000] SBI v0.2 HSM extension detected [ 0.000000] riscv: ISA extensions acimp [ 0.000000] riscv: ELF capabilities acim [ 0.000000] percpu: Embedded 8 pages/cpu s11340 r0 d21428 u32768 [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 32512 [ 0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/ram0 [ 0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes, linear) [ 0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes, linear) [ 0.000000] Sorting __ex_table... [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.000000] Memory: 113620K/131072K available (5685K kernel code, 572K rwdata, 883K rodata, 209K init, 221K bss, 17452K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] riscv-intc: 32 local interrupts mapped [ 0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts. [ 0.000000] random: get_random_bytes called from start_kernel+0x4ac/0x63c with crng_init=0 [ 0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0] [ 0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns [ 0.000124] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns [ 0.015034] Console: colour dummy device 80x25 [ 0.023803] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=192000) [ 0.035914] pid_max: default: 32768 minimum: 301 [ 0.064347] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.073816] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.271613] ASID allocator using 9 bits (512 entries) [ 0.298566] rcu: Hierarchical SRCU implementation. [ 0.356230] smp: Bringing up secondary CPUs ... [ 0.361589] smp: Brought up 1 node, 1 CPU [ 0.413326] devtmpfs: initialized [ 0.675550] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 0.687041] futex hash table entries: 256 (order: 2, 16384 bytes, linear) [ 0.764644] NET: Registered PF_NETLINK/PF_ROUTE protocol family [ 2.286756] pps_core: LinuxPPS API ver. 1 registered [ 2.291895] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti [ 2.303005] PTP clock support registered [ 2.337553] FPGA manager framework [ 2.465054] clocksource: Switched to clocksource riscv_clocksource [ 4.354761] NET: Registered PF_INET protocol family [ 4.376303] IP idents hash table entries: 2048 (order: 2, 16384 bytes, linear) [ 4.447113] tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes, linear) [ 4.462334] TCP established hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 4.474843] TCP bind hash table entries: 1024 (order: 1, 8192 bytes, linear) [ 4.483897] TCP: Hash tables configured (established 1024 bind 1024) [ 4.506406] UDP hash table entries: 256 (order: 1, 8192 bytes, linear) [ 4.518596] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear) [ 4.802981] Unpacking initramfs... [ 5.098350] workingset: timestamp_bits=30 max_order=15 bucket_order=0 [ 7.675652] io scheduler mq-deadline registered [ 7.682910] io scheduler kyber registered [ 11.310904] LiteX SoC Controller driver initialized [ 26.476123] Initramfs unpacking failed: invalid magic at start of compressed archive [ 28.075116] Freeing initrd memory: 8192K [ 32.591960] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart [ 32.606940] printk: console [liteuart0] enabled [ 32.606940] printk: console [liteuart0] enabled [ 32.614654] printk: bootconsole [liteuart0] disabled [ 32.614654] printk: bootconsole [liteuart0] disabled [ 33.018394] i2c_dev: i2c /dev entries driver [ 33.175242] mmc_spi spi0.0: SD/MMC host mmc0, no WP, no poweroff, cd polling [ 33.584093] NET: Registered PF_INET6 protocol family [ 34.551630] Segment Routing with IPv6 [ 34.562796] In-situ OAM (IOAM) with IPv6 [ 34.578854] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver [ 34.727777] NET: Registered PF_PACKET protocol family [ 35.048255] Freeing unused kernel image (initmem) memory: 204K [ 35.058976] Kernel memory protection not selected by kernel config. [ 35.070574] Run /init as init process Starting syslogd: OK Starting klogd: OK Running sysctl: OK Saving random seed: [ 89.918881] random: dd: uninitialized urandom read (512 bytes read) OK Starting network: OK Welcome to Buildroot buildroot login: root __ _ / / (_)__ __ ____ __ / /__/ / _ \/ // /\ \ / /____/_/_//_/\_,_//_\_\ / _ \/ _ \ __ _ __ _ _\___/_//_/ ___ _ / / (_) /____ | |/_/__| | / /____ __ / _ \(_)__ _____ __ / /__/ / __/ -_)>

Maybe it would help you.

peiceliu commented 11 months ago

Thanks bro @avkghost,now i boot linux on my fpga board successfully.

nohahanon commented 11 months ago

@avkghost @peiceliu Big thanks for the great news and sharing it. If you've had success with the Primer 20k, I hope I can achieve the same results with the Nano 20k... However, when it comes to writing the fs file to the board using Gowin and booting with self-built Image, I encountered the same issue as before – the process stops. What steps should I take within the gowin project, including adding mem.init and such? The synthesis process stops due to errors.

serialboot: [LITEX-TERM] Got unexpected response from device 'b'E''
sdcardboot: `litex> sdcardboot

Booting from SDCard in SD-Mode... Booting from boot.json... Copying Image to 0x40000000 (7726264 bytes)... [########################################] Copying rv32.dtb to 0x40ef0000 (2689 bytes)... [########################################] Copying rootfs.cpio to 0x41000000 (5376512 bytes)... [########################################] Copying opensbi.bin to 0x40f00000 (263652 bytes)... [########################################] Executing booted program at 0x40f00000

--============= Liftoff! ===============--`

avkghost commented 11 months ago

@nohahanon As I wrote before, you are still using an incorrect memory map in a bootloader. Maybe you using the wrong address for opensbi/fw_jump for building and/or loading. The second possible problem I see is disabling verification during load into RAM. Your log file leds that rv32.dtb at 0x40ef0000, rootfs.cpio at 0x41000000, and placed at an unreachable memory region. On a Tang Nano 20k is possible to access the memory region from address 0x40000000 to 0x40800000 (default and your variant). All your code/data needs to be placed in this region. For example, the original kernel+root.cpio, opensbi.bin.tangnano20k, and device tree are placed below the memory limit (0x40800000).

All addresses below the memory limit are included in the original example. Please pay attention to the addresses provided below.

{
    "Image":            "0x40000000",
    "sipeed_tang_nano_20k.dtb": "0x40780000",
    "opensbi.bin.tangnano20k":  "0x407c0000"
}

So, I also booted Tang Nano 20k from SDCard.

A boot log from SDCard

``` $ litex_term --speed 3000000 /dev/ttyUSB1 __ _ __ _ __ / / (_) /____ | |/_/ / /__/ / __/ -_)> < /____/_/\__/\__/_/|_| Build your hardware, easily! (c) Copyright 2012-2023 Enjoy-Digital (c) Copyright 2007-2015 M-Labs BIOS CRC passed (b7fd5a52) LiteX git sha1: c0d37665 --=============== SoC ==================-- CPU: VexRiscv SMP-LINUX @ 48MHz BUS: WISHBONE 32-bit @ 4GiB CSR: 32-bit data ROM: 64.0KiB SRAM: 6.0KiB L2: 128B SDRAM: 8.0MiB 32-bit @ 48MT/s (CL-2 CWL-2) MAIN-RAM: 8.0MiB --========== Initialization ============-- Initializing SDRAM @0x40000000... Switching SDRAM to software control. Switching SDRAM to hardware control. Memtest at 0x40000000 (2.0MiB)... Write: 0x40000000-0x40200000 2.0MiB Read: 0x40000000-0x40200000 2.0MiB Memtest OK Memspeed at 0x40000000 (Sequential, 2.0MiB)... Write speed: 18.9MiB/s Read speed: 21.2MiB/s --============== Boot ==================-- Booting from serial... Press Q or ESC to abort boot completely. sL5DdSMmkekro Timeout Booting from SDCard in SD-Mode... Booting from boot.json... Copying Image to 0x40000000 (1818868 bytes)... [########################################] Copying sipeed_tang_nano_20k.dtb to 0x40780000 (2115 bytes)... [########################################] Copying opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)... [########################################] Executing booted program at 0x407c0000 --============= Liftoff! ===============-- OpenSBI v0.8-2-ga9ce3ad ____ _____ ____ _____ / __ \ / ____| _ \_ _| | | | |_ __ ___ _ __ | (___ | |_) || | | | | | '_ \ / _ \ '_ \ \___ \| _ < | | | |__| | |_) | __/ | | |____) | |_) || |_ \____/| .__/ \___|_| |_|_____/|____/_____| | | |_| Platform Name : LiteX / VexRiscv-SMP Platform Features : timer,mfdeleg Platform HART Count : 8 Boot HART ID : 0 Boot HART ISA : rv32imacs BOOT HART Features : pmp,scounteren,mcounteren,time BOOT HART PMP Count : 16 Firmware Base : 0x407c0000 Firmware Size : 120 KB Runtime SBI Version : 0.2 MIDELEG : 0x00000222 MEDELEG : 0x0000b109 [ 0.000000] Linux version 6.4.0-rc1+ (zjs@CL-A-00022) (riscv64-linux-gnu-gcc (Debian 12.2.0-13) 12.2.0, GNU ld (GNU Binutils for Debian) 2.40) #10 Tue May 16 09:20:49 CST 2023 [ 0.000000] earlycon: liteuart0 at I/O port 0x0 (options '') [ 0.000000] Malformed early option 'console' [ 0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '') [ 0.000000] printk: bootconsole [liteuart0] enabled [ 0.000000] OF: reserved mem: 0x407c0000..0x407dffff (128 KiB) map non-reusable opensbi@407c0000 [ 0.000000] Zone ranges: [ 0.000000] Normal [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] SBI specification v0.2 detected [ 0.000000] SBI implementation ID=0x1 Version=0x8 [ 0.000000] SBI TIME extension detected [ 0.000000] SBI IPI extension detected [ 0.000000] SBI RFENCE extension detected [ 0.000000] riscv: base ISA extensions aim [ 0.000000] riscv: ELF capabilities aim [ 0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 [ 0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping off. Total pages: 2032 [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.000000] Memory: 6112K/8192K available (1084K kernel code, 82K rwdata, 110K rodata, 495K init, 49K bss, 2080K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-1, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] riscv-intc: 32 local interrupts mapped [ 0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts. [ 0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0] [ 0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns [ 0.000083] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns [ 0.008171] Console: colour dummy device 80x25 [ 0.011117] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=480000) [ 0.014853] pid_max: default: 4096 minimum: 301 [ 0.019100] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.021835] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.092711] ASID allocator using 9 bits (512 entries) [ 0.123245] devtmpfs: initialized [ 0.233158] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns [ 0.235823] futex hash table entries: 16 (order: -5, 192 bytes, linear) [ 0.288092] platform soc: Fixed dependency cycle(s) with /soc/interrupt-controller@f0c00000 [ 0.368091] clocksource: Switched to clocksource riscv_clocksource [ 1.366526] workingset: timestamp_bits=30 max_order=11 bucket_order=0 [ 2.502946] LiteX SoC Controller driver initialized [ 3.099012] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart [ 3.102855] printk: console [liteuart0] enabled [ 3.102855] printk: console [liteuart0] enabled [ 3.105105] printk: bootconsole [liteuart0] disabled [ 3.105105] printk: bootconsole [liteuart0] disabled [ 3.315436] clk: Disabling unused clocks [ 3.319631] Warning: unable to open an initial console. [ 3.361745] Freeing unused kernel image (initmem) memory: 492K [ 3.363449] Kernel memory protection not selected by kernel config. [ 3.365095] Run /init as init process starting pid 18, tty '/dev/console': '/sbin/getty -L -n -l /bin/sh console 0 vt100' / # ```

To reduce the memory footprint for the kernel try to use make tinyconfigand then add the needed options. For buildroot I recommend using uclibc instead of glibc to reduce the memory footprint.

avkghost commented 11 months ago

@nohahanon Small update Below is my result of the kernel compilation and execution. I removed SMP, networking, and all compressions instead of GZIP. Enabled "Emit compressed instructions when building Linux" in the kernel config.

Kernel: commit ae80e67c6b48bbedcd13db753237a25b3dec8301 (HEAD -> 6.1.0, origin/litex-rebase, origin/6.1.0) buildroot.
.config: linux-on-litex-vexriscv/buildroot/board/litex_vexriscv/linux.config
opensbi.bin.tangnano20k - from example provide by Tang Nano 20k example repository.
sipeed_tang_nano_20k.dtb - A device tree generated by ./make.py provided above.

I had not used any additional scripts or ran make from buildroot. I've created a test build aiming to minimize the kernel.
Please pay attention to the addresses I used in boot.json. Unfortunately, the kernel did not boot properly. I've only created a kernel, not rootfs cpio and integrated it into the Image.

{
    "Image":            "0x40000000",
    "sipeed_tang_nano_20k.dtb": "0x40780000",
    "opensbi.bin.tangnano20k":  "0x407c0000"
}

Boot log

``` __ _ __ _ __ / / (_) /____ | |/_/ / /__/ / __/ -_)> < /____/_/\__/\__/_/|_| Build your hardware, easily! (c) Copyright 2012-2023 Enjoy-Digital (c) Copyright 2007-2015 M-Labs BIOS CRC passed (b7fd5a52) LiteX git sha1: c0d37665 --=============== SoC ==================-- CPU: VexRiscv SMP-LINUX @ 48MHz BUS: WISHBONE 32-bit @ 4GiB CSR: 32-bit data ROM: 64.0KiB SRAM: 6.0KiB L2: 128B SDRAM: 8.0MiB 32-bit @ 48MT/s (CL-2 CWL-2) MAIN-RAM: 8.0MiB --========== Initialization ============-- Initializing SDRAM @0x40000000... Switching SDRAM to software control. Switching SDRAM to hardware control. Memtest at 0x40000000 (2.0MiB)... Write: 0x40000000-0x40200000 2.0MiB Read: 0x40000000-0x40200000 2.0MiB Memtest OK Memspeed at 0x40000000 (Sequential, 2.0MiB)... Write speed: 18.9MiB/s Read speed: 21.2MiB/s --============== Boot ==================-- Booting from serial... Press Q or ESC to abort boot completely. sL5DdSMmkekro [LITEX-TERM] Received firmware download request from the device. [LITEX-TERM] Uploading ./Image to 0x40000000 (3547064 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (131.6KB/s). [LITEX-TERM] Uploading ./sipeed_tang_nano_20k.dtb to 0x40780000 (2689 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (122.0KB/s). [LITEX-TERM] Uploading ./opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (130.1KB/s). [LITEX-TERM] Booting the device. [LITEX-TERM] Done. Executing booted program at 0x407c0000 --============= Liftoff! ===============-- OpenSBI v0.8-2-ga9ce3ad ____ _____ ____ _____ / __ \ / ____| _ \_ _| | | | |_ __ ___ _ __ | (___ | |_) || | | | | | '_ \ / _ \ '_ \ \___ \| _ < | | | |__| | |_) | __/ | | |____) | |_) || |_ \____/| .__/ \___|_| |_|_____/|____/_____| | | |_| Platform Name : LiteX / VexRiscv-SMP Platform Features : timer,mfdeleg Platform HART Count : 8 Boot HART ID : 0 Boot HART ISA : rv32imacs BOOT HART Features : pmp,scounteren,mcounteren,time BOOT HART PMP Count : 16 Firmware Base : 0x407c0000 Firmware Size : 120 KB Runtime SBI Version : 0.2 MIDELEG : 0x00000222 MEDELEG : 0x0000b109 [ 0.000000] Linux version 6.1.0-rc2-296014-gae80e67c6b48 (andy@laptop) (riscv64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #1 Sat Oct 7 23:39:42 CEST 2023 [ 0.000000] earlycon: liteuart0 at I/O port 0x0 (options '') [ 0.000000] Malformed early option 'console' [ 0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '') [ 0.000000] printk: bootconsole [liteuart0] enabled [ 0.000000] INITRD: 0x41000000+0x00800000 is not a memory region - disabling initrd [ 0.000000] Zone ranges: [ 0.000000] Normal [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] SBI specification v0.2 detected [ 0.000000] SBI implementation ID=0x1 Version=0x8 [ 0.000000] SBI TIME extension detected [ 0.000000] SBI IPI extension detected [ 0.000000] SBI RFENCE extension detected [ 0.000000] riscv: base ISA extensions aim [ 0.000000] riscv: ELF capabilities aim [ 0.000000] Built 1 zonelists, mobility grouping off. Total pages: 2032 [ 0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/ram0 [ 0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.000000] Memory: 4364K/8192K available (2361K kernel code, 510K rwdata, 445K rodata, 142K init, 241K bss, 3828K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] riscv-intc: 32 local interrupts mapped [ 0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts. [ 0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0] [ 0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns [ 0.000081] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns [ 0.011053] Console: colour dummy device 80x25 [ 0.014633] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=192000) [ 0.017877] pid_max: default: 4096 minimum: 301 [ 0.035452] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.038862] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.128176] ASID allocator using 9 bits (512 entries) [ 0.290047] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns ```

@Dolu1990 Could I ask you for help with kernel debugging? Maybe I removed something important from the kernel configuration or did something wrong. Thanks.

nohahanon commented 11 months ago

Big thanks, bro, I'll try.

avkghost commented 11 months ago

@nohahanon Another small update I successfully booted up my own custom-built environment.

kernel - as described above
sipeed_tang_nano_20k.dtb - I have made some customizations to the device tree.
root.cpio - extracted from tang nano example kernel image
opensbi.bin.tangnano20k

boot.json

{
    "Image":            "0x40000000",
    "root.cpio":            "0x406ef000",
    "sipeed_tang_nano_20k.dtb": "0x40780000",
    "opensbi.bin.tangnano20k":  "0x407c0000"
}

device tree

``` /dts-v1/; / { #address-cells = <1>; #size-cells = <1>; chosen { bootargs = "console=liteuart earlycon=liteuart,0xf0001000 root=/dev/ram0"; linux,initrd-start = <0x406EF000>; linux,initrd-end = <0x4077FFFF>; }; sys_clk: pll { compatible = "fixed-clock"; #clock-cells = <0>; clock-frequency = <48000000>; }; cpus { #address-cells = <1>; #size-cells = <0>; timebase-frequency = <48000000>; CPU0: cpu@0 { device_type = "cpu"; compatible = "riscv"; riscv,isa = "rv32i2p0_ma"; mmu-type = "riscv,sv32"; reg = <0>; clock-frequency = <48000000>; status = "okay"; d-cache-size = <4096>; d-cache-sets = <1>; d-cache-block-size = <64>; i-cache-size = <4096>; i-cache-sets = <1>; i-cache-block-size = <64>; d-tlb-size = <4>; d-tlb-sets = <4>; i-tlb-size = <4>; i-tlb-sets = <4>; L0: interrupt-controller { #address-cells = <0>; #interrupt-cells = <0x00000001>; interrupt-controller; compatible = "riscv,cpu-intc"; }; }; }; memory@40000000 { device_type = "memory"; reg = <0x40000000 0x800000>; }; reserved-memory { #address-cells = <1>; #size-cells = <1>; ranges; cpio@406ef000 { reg = <0x406ef000 0x91000>; }; opensbi@407c0000 { reg = <0x407c0000 0x80000>; }; }; vreg_mmc: vreg_mmc { compatible = "regulator-fixed"; regulator-name = "vreg_mmc"; regulator-min-microvolt = <3300000>; regulator-max-microvolt = <3300000>; regulator-always-on; }; soc { #address-cells = <1>; #size-cells = <1>; compatible = "simple-bus"; interrupt-parent = <&intc0>; ranges; soc_ctrl0: soc_controller@f0000800 { compatible = "litex,soc-controller"; reg = <0xf0000800 0xc>; status = "okay"; }; intc0: interrupt-controller@f0c00000 { compatible = "sifive,fu540-c000-plic", "sifive,plic-1.0.0"; reg = <0xf0c00000 0x400000>; #address-cells = <0>; #interrupt-cells = <1>; interrupt-controller; interrupts-extended = < &L0 11 &L0 9>; riscv,ndev = <32>; }; liteuart0: serial@f0001000 { compatible = "litex,liteuart"; reg = <0xf0001000 0x100>; interrupts = <1>; status = "okay"; }; mmc0: mmc@f0005000 { compatible = "litex,mmc"; reg = <0xf0005000 0x100>, <0xf0003800 0x100>, <0xf0003000 0x100>, <0xf0004800 0x100>, <0xf0004000 0x100>; reg-names = "phy", "core", "reader", "writer", "irq"; clocks = <&sys_clk>; vmmc-supply = <&vreg_mmc>; bus-width = <0x04>; interrupts = <3>; status = "okay"; }; leds: gpio@f0002800 { compatible = "litex,gpio"; reg = <0xf0002800 0x4>; gpio-controller; #gpio-cells = <2>; litex,direction = "out"; status = "disabled"; }; }; aliases { serial0 = &liteuart0; }; }; &leds { litex,ngpio = <4>; status = "okay"; }; ```

boot log

``` litex_term --speed 3000000 --serial-boot --image ./boot.json /dev/ttyUSB1 __ _ __ _ __ / / (_) /____ | |/_/ / /__/ / __/ -_)> < /____/_/\__/\__/_/|_| Build your hardware, easily! (c) Copyright 2012-2023 Enjoy-Digital (c) Copyright 2007-2015 M-Labs BIOS CRC passed (b7fd5a52) LiteX git sha1: c0d37665 --=============== SoC ==================-- CPU: VexRiscv SMP-LINUX @ 48MHz BUS: WISHBONE 32-bit @ 4GiB CSR: 32-bit data ROM: 64.0KiB SRAM: 6.0KiB L2: 128B SDRAM: 8.0MiB 32-bit @ 48MT/s (CL-2 CWL-2) MAIN-RAM: 8.0MiB --========== Initialization ============-- Initializing SDRAM @0x40000000... Switching SDRAM to software control. Switching SDRAM to hardware control. Memtest at 0x40000000 (2.0MiB)... Write: 0x40000000-0x40200000 2.0MiB Read: 0x40000000-0x40200000 2.0MiB Memtest OK Memspeed at 0x40000000 (Sequential, 2.0MiB)... Write speed: 18.9MiB/s Read speed: 21.2MiB/s --============== Boot ==================-- Booting from serial... Press Q or ESC to abort boot completely. sL5DdSMmkekro [LITEX-TERM] Received firmware download request from the device. [LITEX-TERM] Uploading ./Image to 0x40000000 (2993104 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (132.2KB/s). [LITEX-TERM] Uploading ./root.cpio to 0x406ef000 (592224 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (136.6KB/s). [LITEX-TERM] Uploading ./sipeed_tang_nano_20k.dtb to 0x40780000 (2725 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (115.8KB/s). [LITEX-TERM] Uploading ./opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 320.00us, length: 64) [LITEX-TERM] Upload complete (137.4KB/s). [LITEX-TERM] Booting the device. [LITEX-TERM] Done. Executing booted program at 0x407c0000 --============= Liftoff! ===============-- OpenSBI v0.8-2-ga9ce3ad ____ _____ ____ _____ / __ \ / ____| _ \_ _| | | | |_ __ ___ _ __ | (___ | |_) || | | | | | '_ \ / _ \ '_ \ \___ \| _ < | | | |__| | |_) | __/ | | |____) | |_) || |_ \____/| .__/ \___|_| |_|_____/|____/_____| | | |_| Platform Name : LiteX / VexRiscv-SMP Platform Features : timer,mfdeleg Platform HART Count : 8 Boot HART ID : 0 Boot HART ISA : rv32imacs BOOT HART Features : pmp,scounteren,mcounteren,time BOOT HART PMP Count : 16 Firmware Base : 0x407c0000 Firmware Size : 120 KB Runtime SBI Version : 0.2 MIDELEG : 0x00000222 MEDELEG : 0x0000b109 [ 0.000000] Linux version 6.1.0-rc2-296014-gae80e67c6b48 (andy@laptop) (riscv64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #17 SMP Sun Oct 8 02:25:59 CEST 2023 [ 0.000000] earlycon: liteuart0 at I/O port 0x0 (options '') [ 0.000000] Malformed early option 'console' [ 0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '') [ 0.000000] printk: bootconsole [liteuart0] enabled [ 0.000000] Zone ranges: [ 0.000000] Normal [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] SBI specification v0.2 detected [ 0.000000] SBI implementation ID=0x1 Version=0x8 [ 0.000000] SBI TIME extension detected [ 0.000000] SBI IPI extension detected [ 0.000000] SBI RFENCE extension detected [ 0.000000] SBI HSM extension detected [ 0.000000] riscv: base ISA extensions aim [ 0.000000] riscv: ELF capabilities aim [ 0.000000] percpu: Embedded 8 pages/cpu s10896 r0 d21872 u32768 [ 0.000000] Built 1 zonelists, mobility grouping off. Total pages: 2032 [ 0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 root=/dev/ram0 [ 0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.000000] Memory: 4076K/8192K available (1876K kernel code, 516K rwdata, 379K rodata, 146K init, 232K bss, 4116K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=32 to nr_cpu_ids=1. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] riscv-intc: 32 local interrupts mapped [ 0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts. [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. [ 0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0] [ 0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns [ 0.000100] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns [ 0.014662] Console: colour dummy device 80x25 [ 0.019356] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=192000) [ 0.022018] pid_max: default: 32768 minimum: 301 [ 0.041167] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.045037] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.235037] ASID allocator using 9 bits (512 entries) [ 0.257621] rcu: Hierarchical SRCU implementation. [ 0.260791] rcu: Max phase no-delay instances is 1000. [ 0.317495] smp: Bringing up secondary CPUs ... [ 0.318840] smp: Brought up 1 node, 1 CPU [ 0.361797] devtmpfs: initialized [ 0.555579] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 0.559983] futex hash table entries: 256 (order: 2, 16384 bytes, linear) [ 0.695002] cpuidle: using governor ladder [ 1.883589] FPGA manager framework [ 1.900723] clocksource: Switched to clocksource riscv_clocksource [ 3.332691] Unpacking initramfs... [ 3.364459] workingset: timestamp_bits=30 max_order=10 bucket_order=0 [ 6.694820] Initramfs unpacking failed: broken padding [ 6.764988] Freeing initrd memory: 576K [ 6.944560] LiteX SoC Controller driver initialized [ 14.876584] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart [ 14.883637] printk: console [liteuart0] enabled [ 14.883637] printk: console [liteuart0] enabled [ 14.887804] printk: bootconsole [liteuart0] disabled [ 14.887804] printk: bootconsole [liteuart0] disabled [ 15.039121] i2c_dev: i2c /dev entries driver [ 15.059933] cpuidle-riscv-sbi: HSM suspend not available [ 15.188386] litex-mmc f0005000.mmc: LiteX MMC controller initialized. [ 15.351267] Freeing unused kernel image (initmem) memory: 140K [ 15.354801] Kernel memory protection not selected by kernel config. [ 15.356935] Run /init as init process [ 15.533254] mmc0: new SDXC card at address aaaa starting pid 24, tty '': '/etc/init.d/rcS' [ 16.547433] tmpfs: Unknown parameter 'mode' mount: mounting tmpfs on /dev/shm failed: Invalid argument [ 16.596831] tmpfs: Unknown parameter 'mode' mount: mounting tmpfs on /tmp failed: Invalid argument starting pid 28, tty '/dev/console': '/sbin/getty -L -n -l /bin/sh console 0 vt100' / # free total used free shared buff/cache available Mem: 4792 1776 2032 0 984 1868 Swap: 0 0 0 ```

nohahanon commented 11 months ago

@avkghost

Thank you for your reply. I edited linux.config like this and I successfully reduced the size to less than 8MB! In the comment two comments back, you were saying I removed SMP, networking, and all compressions instead of GZIP. Enabled "Emit compressed instructions when building Linux" in the kernel config., but did you mean that you edited the linux.config file like below?

CONFIG_RISCV_ISA_C=n -> y
CONFIG_SMP=y -> n
CONFIG_RD_BZIP2=y -> n
CONFIG_RD_LZMA=y -> n
CONFIG_RD_XZ=y -> n
CONFIG_RD_LZO=y -> n
CONFIG_RD_LZ4=y -> n
Ethernet config -> n

In conclusion, I failed to boot linux using the following file. It stopped in the middle of the opensbi process. By the way, you said root.cpio - extracted from tang nano example kernel image, could you please explain how to do that?

Regards.

linux.config

``` CONFIG_SECTION_MISMATCH_WARN_ONLY=y # Architecture CONFIG_ARCH_DEFCONFIG="arch/riscv/configs/defconfig" CONFIG_NONPORTABLE=y CONFIG_ARCH_RV32I=y CONFIG_RISCV_ISA_M=y CONFIG_RISCV_ISA_A=y CONFIG_RISCV_ISA_C=y CONFIG_SIFIVE_PLIC=y CONFIG_FPU=n CONFIG_SMP=n CONFIG_STRICT_KERNEL_RWX=n CONFIG_EFI=n CONFIG_HVC_RISCV_SBI=y CONFIG_NONPORTABLE=y CONFIG_BLK_DEV_INITRD=y CONFIG_INITRAMFS_SOURCE="" CONFIG_RD_GZIP=y CONFIG_RD_BZIP2=n CONFIG_RD_LZMA=n CONFIG_RD_XZ=n CONFIG_RD_LZO=n CONFIG_RD_LZ4=n # FPGA / SoC CONFIG_FPGA=y CONFIG_FPGA_MGR_LITEX=y CONFIG_LITEX_SOC_CONTROLLER=y CONFIG_LITEX_SUBREG_SIZE=4 # Time CONFIG_PRINTK_TIME=y # Clocking CONFIG_COMMON_CLK=y CONFIG_COMMON_CLK_LITEX=y # Interrupts CONFIG_IRQCHIP=y CONFIG_OF_IRQ=y CONFIG_HANDLE_DOMAIN_IRQ=y CONFIG_LITEX_VEXRISCV_INTC=y # Ethernet CONFIG_NET=n CONFIG_PACKET=n CONFIG_PACKET_DIAG=n CONFIG_INET=n CONFIG_NETDEVICES=n CONFIG_NET_VENDOR_LITEX=n CONFIG_LITEX_LITEETH=n # Serial CONFIG_SERIAL_EARLYCON_RISCV_SBI=y CONFIG_SERIAL_LITEUART=y CONFIG_SERIAL_LITEUART_CONSOLE=y # GPIO CONFIG_GPIO_SYSFS=y CONFIG_GPIOLIB=y CONFIG_GPIO_LITEX=y # PWM CONFIG_PWM=y CONFIG_PWM_LITEX=y # SPI CONFIG_SPI=y CONFIG_SPI_LITESPI=y CONFIG_SPI_SPIDEV=y # I2C CONFIG_I2C=y CONFIG_I2C_LITEX=y CONFIG_I2C_CHARDEV=y # Hardware monitoring CONFIG_HWMON=y CONFIG_SENSORS_LITEX_HWMON=y # Framebuffer CONFIG_FB=y CONFIG_FB_SIMPLE=y CONFIG_FRAMEBUFFER_CONSOLE=y CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=y CONFIG_LOGO=y CONFIG_DRM=y CONFIG_DRM_LITEVIDEO=y # Flash CONFIG_MTD=y CONFIG_MTD_SPI_NOR=y CONFIG_SPI_FLASH_LITEX=y # MMC CONFIG_MMC=y CONFIG_MMC_SPI=y CONFIG_MMC_LITEX=y CONFIG_EXT2_FS=y CONFIG_EXT3_FS=y CONFIG_EXT4_FS=y # .config in kernel CONFIG_IKCONFIG=y CONFIG_IKCONFIG_PROC=y # Filesystem CONFIG_FAT_FS=y CONFIG_MSDOS_FS=y CONFIG_MSDOS_PARTITION=y CONFIG_VFAT_FS=y CONFIG_FAT_DEFAULT_CODEPAGE=437 CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1" CONFIG_NCPFS_SMALLDOS=y CONFIG_NLS=y CONFIG_NLS_CODEPAGE_437=y CONFIG_NLS_ASCII=y CONFIG_NLS_ISO8859_1=y CONFIG_TMPFS=y ```

size of each file

Image: 4125780(0x3EF454), same as you
opensbi.bin.tangnano20k: 49636(0xC1E4), example's one
rootfs.cpio: 3712512(0x38A600), created simultaneously with the image
sipeed_tang_nano_20k.dtb: 2725(AA5), compiled version of the following .dts
total: 7890653 < 8MB!!!

boot.json

{
    "Image":            "0x40000000",
    "rootfs.cpio":          "0x40400000",
    "sipeed_tang_nano_20k.dtb": "0x40780000",
    "opensbi.bin.tangnano20k":  "0x407c0000"
}

devise tree

``` /dts-v1/; / { #address-cells = <1>; #size-cells = <1>; chosen { bootargs = "console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/ram0"; linux,initrd-start = <0x41000000>; linux,initrd-end = <0x41800000>; }; sys_clk: pll { compatible = "fixed-clock"; #clock-cells = <0>; clock-frequency = <48000000>; }; cpus { #address-cells = <1>; #size-cells = <0>; timebase-frequency = <48000000>; CPU0: cpu@0 { device_type = "cpu"; compatible = "riscv"; riscv,isa = "rv32i2p0_ma"; mmu-type = "riscv,sv32"; reg = <0>; clock-frequency = <48000000>; status = "okay"; d-cache-size = <4096>; d-cache-sets = <1>; d-cache-block-size = <64>; i-cache-size = <4096>; i-cache-sets = <1>; i-cache-block-size = <64>; d-tlb-size = <4>; d-tlb-sets = <4>; i-tlb-size = <4>; i-tlb-sets = <4>; L0: interrupt-controller { #address-cells = <0>; #interrupt-cells = <0x00000001>; interrupt-controller; compatible = "riscv,cpu-intc"; }; }; }; memory@40000000 { device_type = "memory"; reg = <0x40000000 0x800000>; }; reserved-memory { #address-cells = <1>; #size-cells = <1>; ranges; cpio@40400000 { reg = <0x40400000 0x91000>; }; opensbi@407c0000 { reg = <0x407c0000 0x80000>; }; }; vreg_mmc: vreg_mmc { compatible = "regulator-fixed"; regulator-name = "vreg_mmc"; regulator-min-microvolt = <3300000>; regulator-max-microvolt = <3300000>; regulator-always-on; }; soc { #address-cells = <1>; #size-cells = <1>; compatible = "simple-bus"; interrupt-parent = <&intc0>; ranges; soc_ctrl0: soc_controller@f0000800 { compatible = "litex,soc-controller"; reg = <0xf0000800 0xc>; status = "okay"; }; intc0: interrupt-controller@f0c00000 { compatible = "sifive,fu540-c000-plic", "sifive,plic-1.0.0"; reg = <0xf0c00000 0x400000>; #address-cells = <0>; #interrupt-cells = <1>; interrupt-controller; interrupts-extended = < &L0 11 &L0 9>; riscv,ndev = <32>; }; liteuart0: serial@f0001000 { compatible = "litex,liteuart"; reg = <0xf0001000 0x100>; interrupts = <1>; status = "okay"; }; mmc0: mmc@f0005000 { compatible = "litex,mmc"; reg = <0xf0005000 0x100>, <0xf0003800 0x100>, <0xf0003000 0x100>, <0xf0004800 0x100>, <0xf0004000 0x100>; reg-names = "phy", "core", "reader", "writer", "irq"; clocks = <&sys_clk>; vmmc-supply = <&vreg_mmc>; bus-width = <0x04>; interrupts = <3>; status = "okay"; }; leds: gpio@f0002800 { compatible = "litex,gpio"; reg = <0xf0002800 0x4>; gpio-controller; #gpio-cells = <2>; litex,direction = "out"; status = "disabled"; }; }; aliases { serial0 = &liteuart0; }; }; &leds { litex,ngpio = <4>; status = "okay"; }; ```

boot log

``` Booting from serial... Press Q or ESC to abort boot completely. sL5DdSMmkekro [LITEX-TERM] Received firmware download request from the device. [LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/Image to 0x40000000 (4125780 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 20.00us, length: 64) [LITEX-TERM] Upload complete (260.0KB/s). [LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/rootfs.cpio to 0x40400000 (3712512 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Upload complete (255.3KB/s). [LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/sipeed_tang_nano_20k.dtb to 0x40780000 (2733 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Upload complete (143.8KB/s). [LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Upload complete (189.8KB/s). [LITEX-TERM] Booting the device. [LITEX-TERM] Done. Executing booted program at 0x407c0000 --============= Liftoff! ===============-- OpenSBI v0.8-2-ga9ce3ad ____ _____ ____ _____ / __ \ / ____| _ \_ _| | | | |_ __ ___ _ __ | (___ | |_) || | | | | | '_ \ / _ \ '_ \ \___ \| _ < | | | |__| | |_) | __/ | | |____) | |_) || |_ \____/| .__/ \___|_| |_|_____/|____/_____| | | |_| Platform Name : LiteX / VexRiscv-SMP Platform Features : timer,mfdeleg Platform HART Count : 8 Boot HART ID : 0 Boot HART ISA : rv32imas BOOT HART Features : time BOOT HART PMP Count : 0 Firmware Base : 0x407c0000 Firmware Size : 120 KB Runtime SBI Version : 0.2 MIDELEG : 0x00000222 MEDELEG : 0x0000b101 (halt...) ```

avkghost commented 11 months ago

@nohahanon

linux.config

An "Emit compressed instructions when building Linux" option is CONFIG_RISCV_ISA_C=y

CONFIG_SMP=n

Yes, it disables SMP.

Disable networking CONFIG_NET=n or remove it.

Image: 4125780(0x3EF454), same as you

So close, but mine is 2993104 bytes.

root.cpio

In conclusion, I failed to boot linux using the following file. It stopped in the middle of the opensbi process. By the way, you said root.cpio - extracted from tang nano example kernel image, could you please explain how to do that?

I used binwalk to extrat root.cpio from the example's kernel. binwalk -e Image. An extracted cpio was placed in _Image.extracted directory.

offtopic

I had not used buildroot for some reasons:

Separate kernel configuration such as make config or make menuconfig gives me a fine kernel tuning.
A buildroot provided in a [recipe] (https://github.com/litex-hub/linux-on-litex-vexriscv#-generating-the-linux-binaries-optional) didn't work for me. *** No C library enabled, this is not possible.. Stop. Maybe it depens on some addition parts like compiler I don't know. I have a riscv coross-compiler + binutils installed from apt.

nohahanon commented 11 months ago

@avkghost I hadn't know binwalk! Thank you very much.

Currently, I believe the only difference between your setup and mine is the Image. Could you please share how you created that Image?

By the way, I encountered the same error with the provided Buildroot recipe, but when I extracted the tar.gz available on buildroot.org and did the make process, I was able to obtain the Image without any errors. So, I am using the Image created using this method.

I'm sorry for taking up your time. Thank you.

report Even with opensbi processing, it progressed quite far, but it still didn't reach the boot and stopped.

When I replaced the rootfs.cpio extracted using binwalk from the example Image and executed it, opensbi processing stopped in the middle.

boot.json

``` { "Image": "0x40000000", "rootfs.cpio": "0x4050FFFF", "sipeed_tang_nano_20k.dtb": "0x40780000", "opensbi.bin.tangnano20k": "0x407c0000" } ```

.dts

``` /dts-v1/; / { #address-cells = <1>; #size-cells = <1>; chosen { bootargs = "console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/ram0"; linux,initrd-start = <0x41000000>; linux,initrd-end = <0x41800000>; }; sys_clk: pll { compatible = "fixed-clock"; #clock-cells = <0>; clock-frequency = <48000000>; }; cpus { #address-cells = <1>; #size-cells = <0>; timebase-frequency = <48000000>; CPU0: cpu@0 { device_type = "cpu"; compatible = "riscv"; riscv,isa = "rv32i2p0_ma"; mmu-type = "riscv,sv32"; reg = <0>; clock-frequency = <48000000>; status = "okay"; d-cache-size = <4096>; d-cache-sets = <1>; d-cache-block-size = <64>; i-cache-size = <4096>; i-cache-sets = <1>; i-cache-block-size = <64>; d-tlb-size = <4>; d-tlb-sets = <4>; i-tlb-size = <4>; i-tlb-sets = <4>; L0: interrupt-controller { #address-cells = <0>; #interrupt-cells = <0x00000001>; interrupt-controller; compatible = "riscv,cpu-intc"; }; }; }; memory@40000000 { device_type = "memory"; reg = <0x40000000 0x800000>; }; reserved-memory { #address-cells = <1>; #size-cells = <1>; ranges; cpio@4050FFFF { reg = <0x4050FFFF 0x91000>; }; opensbi@407c0000 { reg = <0x407c0000 0x20000>; }; }; vreg_mmc: vreg_mmc { compatible = "regulator-fixed"; regulator-name = "vreg_mmc"; regulator-min-microvolt = <3300000>; regulator-max-microvolt = <3300000>; regulator-always-on; }; soc { #address-cells = <1>; #size-cells = <1>; compatible = "simple-bus"; interrupt-parent = <&intc0>; ranges; soc_ctrl0: soc_controller@f0000800 { compatible = "litex,soc-controller"; reg = <0xf0000800 0xc>; status = "okay"; }; intc0: interrupt-controller@f0c00000 { compatible = "sifive,fu540-c000-plic", "sifive,plic-1.0.0"; reg = <0xf0c00000 0x400000>; #address-cells = <0>; #interrupt-cells = <1>; interrupt-controller; interrupts-extended = < &L0 11 &L0 9>; riscv,ndev = <32>; }; liteuart0: serial@f0001000 { compatible = "litex,liteuart"; reg = <0xf0001000 0x100>; interrupts = <1>; status = "okay"; }; mmc0: mmc@f0005000 { compatible = "litex,mmc"; reg = <0xf0005000 0x100>, <0xf0003800 0x100>, <0xf0003000 0x100>, <0xf0004800 0x100>, <0xf0004000 0x100>; reg-names = "phy", "core", "reader", "writer", "irq"; clocks = <&sys_clk>; vmmc-supply = <&vreg_mmc>; bus-width = <0x04>; interrupts = <3>; status = "okay"; }; leds: gpio@f0002800 { compatible = "litex,gpio"; reg = <0xf0002800 0x4>; gpio-controller; #gpio-cells = <2>; litex,direction = "out"; status = "disabled"; }; }; aliases { serial0 = &liteuart0; }; }; &leds { litex,ngpio = <4>; status = "okay"; }; ```

boot log

``` litex> serialboot Booting from serial... Press Q or ESC to abort boot completely. sL5DdSMmkekro [LITEX-TERM] Received firmware download request from the device. [LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/Image to 0x40000000 (5305428 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Upload complete (256.4KB/s). [LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/rootfs.cpio to 0x4050ffff (592224 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Upload complete (254.6KB/s). [LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/sipeed_tang_nano_20k.dtb to 0x40780000 (2733 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Upload complete (110.4KB/s). [LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Upload complete (208.8KB/s). [LITEX-TERM] Booting the device. [LITEX-TERM] Done. Executing booted program at 0x407c0000 --============= Liftoff! ===============-- OpenSBI v0.8-2-ga9ce3ad ____ _____ ____ _____ / __ \ / ____| _ \_ _| | | | |_ __ ___ _ __ | (___ | |_) || | | | | | '_ \ / _ \ '_ \ \___ \| _ < | | | |__| | |_) | __/ | | |____) | |_) || |_ \____/| .__/ \___|_| |_|_____/|____/_____| | | |_| Platform Name : LiteX / VexRiscv-SMP Platform Features : timer,mfdeleg Platform HART Count : 8 Boot HART ID : 0 Boot HART ISA : rv32imas BOOT HART Features : time BOOT HART PMP Count : 0 Firmware Base : 0x407c0000 Firmware Size : 120 KB Runtime SBI Version : 0.2 MIDELEG : 0x00000222 MEDELEG : 0x0000b101 ```

When I changed CONFIG_RISCV_ISA_C=n and executed it, opensbi processing stopped at a position further than before! Errors like below are occured...

[    8.899980] litex-mmc f0005000.mmc: Command (cmd 8) error, status -110
[    8.924722] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110
[    8.952200] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110
[    8.976205] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110
[    9.000210] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110

boot.json: same as above
.dts: same as above

boot log

``` litex> serialboot Booting from serial... Press Q or ESC to abort boot completely. sL5DdSMmkekro [LITEX-TERM] Received firmware download request from the device. [LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/Image to 0x40000000 (5305428 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Upload complete (256.4KB/s). [LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/rootfs.cpio to 0x4050ffff (592224 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Upload complete (254.6KB/s). [LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/sipeed_tang_nano_20k.dtb to 0x40780000 (2733 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Upload complete (110.4KB/s). [LITEX-TERM] Uploading ./TangNano-20K-example/linux/image2/opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)... [LITEX-TERM] Upload calibration... (inter-frame: 10.00us, length: 64) [LITEX-TERM] Upload complete (208.8KB/s). [LITEX-TERM] Booting the device. [LITEX-TERM] Done. Executing booted program at 0x407c0000 --============= Liftoff! ===============-- OpenSBI v0.8-2-ga9ce3ad ____ _____ ____ _____ / __ \ / ____| _ \_ _| | | | |_ __ ___ _ __ | (___ | |_) || | | | | | '_ \ / _ \ '_ \ \___ \| _ < | | | |__| | |_) | __/ | | |____) | |_) || |_ \____/| .__/ \___|_| |_|_____/|____/_____| | | |_| Platform Name : LiteX / VexRiscv-SMP Platform Features : timer,mfdeleg Platform HART Count : 8 Boot HART ID : 0 Boot HART ISA : rv32imas BOOT HART Features : time BOOT HART PMP Count : 0 Firmware Base : 0x407c0000 Firmware Size : 120 KB Runtime SBI Version : 0.2 MIDELEG : 0x00000222 MEDELEG : 0x0000b101 [ 0.000000] Linux version 6.1.0-rc2 (root@nohara-MacBookPro) (riscv32-buildroot-linux-gnu-gcc.br_real (Buildroot 2023.02.4) 11.4.0, GNU ld (GNU Binutils) 2.38) #5 Mon Oct 9 10:00:31 JST 2023 [ 0.000000] earlycon: liteuart0 at I/O port 0x0 (options '') [ 0.000000] Malformed early option 'console' [ 0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '') [ 0.000000] printk: bootconsole [liteuart0] enabled [ 0.000000] INITRD: 0x41000000+0x00800000 is not a memory region - disabling initrd [ 0.000000] Zone ranges: [ 0.000000] Normal [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] SBI specification v0.2 detected [ 0.000000] SBI implementation ID=0x1 Version=0x8 [ 0.000000] SBI TIME extension detected [ 0.000000] SBI IPI extension detected [ 0.000000] SBI RFENCE extension detected [ 0.000000] riscv: base ISA extensions aim [ 0.000000] riscv: ELF capabilities aim [ 0.000000] Built 1 zonelists, mobility grouping off. Total pages: 2032 [ 0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/ram0 [ 0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.000000] Memory: 2180K/8192K available (3796K kernel code, 526K rwdata, 694K rodata, 159K init, 235K bss, 6012K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] riscv-intc: 32 local interrupts mapped [ 0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts. [ 0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0] [ 0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns [ 0.000099] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns [ 0.009903] Console: colour dummy device 80x25 [ 0.013447] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=192000) [ 0.015789] pid_max: default: 32768 minimum: 301 [ 0.032100] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.034394] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.134036] ASID allocator using 9 bits (512 entries) [ 0.172003] devtmpfs: initialized [ 0.302573] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 0.305876] futex hash table entries: 256 (order: -1, 3072 bytes, linear) [ 0.894239] FPGA manager framework [ 0.902612] clocksource: Switched to clocksource riscv_clocksource [ 2.013982] workingset: timestamp_bits=30 max_order=10 bucket_order=0 [ 3.127995] io scheduler mq-deadline registered [ 3.129392] io scheduler kyber registered [ 3.237980] LiteX SoC Controller driver initialized [ 8.555817] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart [ 8.560177] printk: console [liteuart0] enabled [ 8.560177] printk: console [liteuart0] enabled [ 8.562360] printk: bootconsole [liteuart0] disabled [ 8.562360] printk: bootconsole [liteuart0] disabled [ 8.664773] i2c_dev: i2c /dev entries driver [ 8.772213] litex-mmc f0005000.mmc: LiteX MMC controller initialized. [ 8.891705] Waiting for root device /dev/ram0... [ 8.899980] litex-mmc f0005000.mmc: Command (cmd 8) error, status -110 [ 8.924722] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110 [ 8.952200] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110 [ 8.976205] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110 [ 9.000210] litex-mmc f0005000.mmc: Command (cmd 55) error, status -110 ```

avkghost commented 11 months ago

@nohahanon I've successfully booted from SDCard and added SDCard support to the kernel. As a root.cpio I continue using the old one from the example above. However, I've successfully built root.cpio with a buildroot, but not used it due to a big footprint and out-of-memory while booting.

A linux.config

Boot log

``` __ _ __ _ __ / / (_) /____ | |/_/ / /__/ / __/ -_)> < /____/_/\__/\__/_/|_| Build your hardware, easily! (c) Copyright 2012-2023 Enjoy-Digital (c) Copyright 2007-2015 M-Labs BIOS CRC passed (b7fd5a52) LiteX git sha1: c0d37665 --=============== SoC ==================-- CPU: VexRiscv SMP-LINUX @ 48MHz BUS: WISHBONE 32-bit @ 4GiB CSR: 32-bit data ROM: 64.0KiB SRAM: 6.0KiB L2: 128B SDRAM: 8.0MiB 32-bit @ 48MT/s (CL-2 CWL-2) MAIN-RAM: 8.0MiB --========== Initialization ============-- Initializing SDRAM @0x40000000... Switching SDRAM to software control. Switching SDRAM to hardware control. Memtest at 0x40000000 (2.0MiB)... Write: 0x40000000-0x40200000 2.0MiB Read: 0x40000000-0x40200000 2.0MiB Memtest OK Memspeed at 0x40000000 (Sequential, 2.0MiB)... Write speed: 18.9MiB/s Read speed: 21.2MiB/s --============== Boot ==================-- Booting from serial... Press Q or ESC to abort boot completely. sL5DdSMmkekro Timeout Booting from SDCard in SD-Mode... Booting from boot.json... Copying Image to 0x40000000 (3337720 bytes)... [########################################] Copying root-new.cpio.gz to 0x406ef000 (216670 bytes)... [########################################] Copying sipeed_tang_nano_20k.dtb to 0x40780000 (2737 bytes)... [########################################] Copying opensbi.bin.tangnano20k to 0x407c0000 (49636 bytes)... [########################################] Executing booted program at 0x407c0000 --============= Liftoff! ===============-- OpenSBI v0.8-2-ga9ce3ad ____ _____ ____ _____ / __ \ / ____| _ \_ _| | | | |_ __ ___ _ __ | (___ | |_) || | | | | | '_ \ / _ \ '_ \ \___ \| _ < | | | |__| | |_) | __/ | | |____) | |_) || |_ \____/| .__/ \___|_| |_|_____/|____/_____| | | |_| Platform Name : LiteX / VexRiscv-SMP Platform Features : timer,mfdeleg Platform HART Count : 8 Boot HART ID : 0 Boot HART ISA : rv32imacs BOOT HART Features : pmp,scounteren,mcounteren,time BOOT HART PMP Count : 16 Firmware Base : 0x407c0000 Firmware Size : 120 KB Runtime SBI Version : 0.2 MIDELEG : 0x00000222 MEDELEG : 0x0000b109 [ 0.000000] Linux version 6.1.0-rc2-296014-gae80e67c6b48 (andy@laptop) (riscv64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #34 PREEMPT Sun Oct 8 18:52:39 CEST 2023 [ 0.000000] earlycon: liteuart0 at I/O port 0x0 (options '') [ 0.000000] Malformed early option 'console' [ 0.000000] earlycon: liteuart0 at MMIO 0xf0001000 (options '') [ 0.000000] printk: bootconsole [liteuart0] enabled [ 0.000000] Zone ranges: [ 0.000000] Normal [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000407fffff] [ 0.000000] SBI specification v0.2 detected [ 0.000000] SBI implementation ID=0x1 Version=0x8 [ 0.000000] SBI TIME extension detected [ 0.000000] SBI IPI extension detected [ 0.000000] SBI RFENCE extension detected [ 0.000000] riscv: base ISA extensions aim [ 0.000000] riscv: ELF capabilities aim [ 0.000000] Built 1 zonelists, mobility grouping off. Total pages: 2032 [ 0.000000] Kernel command line: console=liteuart earlycon=liteuart,0xf0001000 rootwait root=/dev/mmcblk0p2 [ 0.000000] Dentry cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.000000] Inode-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.000000] Memory: 3744K/8192K available (2057K kernel code, 518K rwdata, 569K rodata, 110K init, 228K bss, 4448K reserved, 0K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] rcu: Preemptible hierarchical RCU implementation. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] riscv-intc: 32 local interrupts mapped [ 0.000000] plic: interrupt-controller@f0c00000: mapped 32 interrupts with 1 handlers for 2 contexts. [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. [ 0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0] [ 0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns [ 0.000099] sched_clock: 64 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns [ 0.013174] Console: colour dummy device 80x25 [ 0.016010] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=192000) [ 0.019874] pid_max: default: 32768 minimum: 301 [ 0.038670] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.042434] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.180385] ASID allocator using 9 bits (512 entries) [ 0.194591] rcu: Hierarchical SRCU implementation. [ 0.195837] rcu: Max phase no-delay instances is 1000. [ 0.238191] devtmpfs: initialized [ 0.417723] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 0.420324] futex hash table entries: 256 (order: -1, 3072 bytes, linear) [ 2.067058] FPGA manager framework [ 2.079990] clocksource: Switched to clocksource riscv_clocksource [ 3.505609] Unpacking initramfs... [ 3.565922] workingset: timestamp_bits=30 max_order=10 bucket_order=0 [ 3.794059] Initramfs unpacking failed: no cpio magic [ 3.919034] Freeing initrd memory: 576K [ 5.226289] io scheduler mq-deadline registered [ 5.227708] io scheduler kyber registered [ 5.357527] LiteX SoC Controller driver initialized [ 6.046900] f0001000.serial: ttyLXU0 at MMIO 0x0 (irq = 0, base_baud = 0) is a liteuart [ 6.053728] printk: console [liteuart0] enabled [ 6.053728] printk: console [liteuart0] enabled [ 6.057051] printk: bootconsole [liteuart0] disabled [ 6.057051] printk: bootconsole [liteuart0] disabled [ 6.271572] null_blk: disk nullb0 created [ 6.274070] null_blk: module loaded [ 6.310911] i2c_dev: i2c /dev entries driver [ 6.437830] litex-mmc f0005000.mmc: LiteX MMC controller initialized. [ 6.529881] mmc0: new SDXC card at address aaaa [ 6.542703] Waiting for root device /dev/mmcblk0p2... [ 6.605616] mmcblk0: mmc0:aaaa SD64G 59.5 GiB [ 6.757951] mmcblk0: p1 p2 [ 8.713423] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Quota mode: disabled. [ 8.718849] VFS: Mounted root (ext4 filesystem) readonly on device 179:2. [ 8.755073] devtmpfs: mounted [ 8.770224] Freeing unused kernel image (initmem) memory: 108K [ 8.773296] Kernel memory protection not selected by kernel config. [ 8.775167] Run /sbin/init as init process starting pid 27, tty '': '/etc/init.d/rcS' mount: mounting devtmpfs on /dev failed: Resource busy starting pid 31, tty '/dev/console': '/sbin/getty -L -n -l /bin/sh console 0 vt100' / # uname -a Linux (none) 6.1.0-rc2-296014-gae80e67c6b48 #34 PREEMPT Sun Oct 8 18:52:39 CEST 2023 riscv32 GNU/Linux / # df -h Filesystem Size Used Available Use% Mounted on /dev/root 57.8G 476.0K 54.8G 0% / devtmpfs 1.8M 0 1.8M 0% /dev tmpfs 2.2M 0 2.2M 0% /dev/shm tmpfs 2.2M 0 2.2M 0% /tmp / # ```

nohahanon commented 11 months ago

@avkghost Is that true! I'm glad to hear that report. My goal is to bring files I've created locally onto that Linux running on tang nano 20k so I can execute a C program. However, I couldn't mount it on the SD card in the Linux image provided in the tang nano 20k example (it seems to be a very minimal Linux). So, I want to create my own, more feature-rich image and use it to boot Linux.

You mentioned that you haven't created an image with Buildroot, but how did you create it? I'd like to create an image similar to what you're using. Looking forward to your reply.

Thanks.

avkghost commented 11 months ago

@nohahanon Initially, I had a problem creating images with build root. For now, I've successfully built it but I need to tune the size due to an out-of-memory error while Linux loading.

I prefer to use separate kernel configurations to minimize its footprint. I cloned the litex-linux repo and checked out 6.1.0 branch, and used the config provided above to build the kernel image.

To cross-compile build the kernel I used the following command. make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu-

Below, is a device tree I've used dt_nano_20k.zip

avkghost commented 11 months ago

@nohahanon Below is an archive with my experiments. nano_examples.zip It is possible to boot them via serial or SD card.

The SD card needs to be partitioned into 2 or 3 partitions.

partition 1 (mmcblk0p1) - 200MB, FAT (For experiments I prefer to use 500MB). It is possible to use a small size.
partition 2 (mmcblk0p2) - as many as you need, /, Ext4 (I've added only ext4 support into the kernel).
partition 3 (mmcblk0p3) - as many as you need, swap. I do not use it.

A command line from the device tree sets mmcblk0p2 as the root partition. The root partition I extracted from root.cpio.

I couldn't mount it on the SD card in the Linux image provided in the tang nano 20k example (it seems to be a very minimal Linux).

The kernel provided in the Tang Nano example did not have drivers for MMC, block devices, partition schemes, and file systems. It is impossible to mount any filesystem under the example's kernel. It is built as an experiment to show the possibility of running Linux on the board.

Now, I have no idea where I can use this configuration. In other words, I don't see any practical use for this configuration in my projects. In my opinion, there is the best way to use this board with bare metal OS, like FreeRTOS, etc, because in this case, Linux is an overhead for the board. For me, it is interesting in the case of porting/running Linux to embedding systems with an architecture different than x86.

If you need to build and run your own program on a device I recommend you start with litex_bare_metal_demo Build your own demo and run it on the board. For example:

create a bitstream and bios if needed. When the script builds the BIOS it also creates a tiny library for I/O that possible to use in self-made projects.
run litex_bare_metal_demo --with-cxx --build-path build/sipeed_tang_nano_20k in a directory contains build result. It creates a demo that you can run on a device
run litex_term --kernel demo/demo.bin and see how it works.

Hope you find it helpful.

nohahanon commented 11 months ago

Big thanks, bro,

I was able to successfully boot Linux using the provided image, but it seems that I still couldn't mount the SD card. While my initial goal was to run a program on Linux, I'm glad that I was able to run programs like Donut and Helloc on VexRiscv　with litex_sim. Although I didn't achieve the original goal of running a C program on Linux booted with the locally built Linux image, I will continue this on an Arty board where there is no need to reduce the size of the image.

litex-hub / linux-on-litex-vexriscv