search

scanakci / linux-on-litex-blackparrot

Linux on Litex for BlackParrot Core
7 stars 4 forks source link

Missing files and errors on simulation start #1

Closed troibe closed 3 years ago

troibe commented 3 years ago

When I'm trying to run the simulation it seems like a lot of files are missing in pythondata-cpu-blackparrot. Which version of blackparrot were you basing your project on? I'm using the master branch of pythondata-cpu-blackparrot currently.

For context I'm trying to first get to run your existing litex-blackparrot project and then update it to a more recent version of blackparrot.

This is the output that I'm getting currently:

%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_be/src/v/bp_be_calculator/bp_be_pipe_long.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_me/src/v/cce/bp_cce_arbitrate.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_me/src/v/cce/bp_cce_branch.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_me/src/v/cce/bp_cce_dir_segment.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_me/src/v/cce/bp_cce_inst_predecode.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_me/src/v/cce/bp_cce_inst_ram.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_me/src/v/cce/bp_cce_inst_stall.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_me/src/v/cce/bp_cce_pending_bits.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_me/src/v/cce/bp_cce_spec_bits.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_me/src/v/cce/bp_cce_src_sel.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_me/src/v/cce/bp_cce_wrapper.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_nd_socket.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_cacc_vdp.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_cacc_tile.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_cacc_tile_node.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_cacc_complex.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_sacc_vdp.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_sacc_tile.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_sacc_tile_node.v
%Error: Cannot find file containing module: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_sacc_complex.v
%Error: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_cfg.v:38:5: syntax error, unexpected IDENTIFIER, expecting TYPE-IDENTIFIER
   38 |     bp_cce_mem_req_size_e                        size;          
      |     ^~~~~~~~~~~~~~~~~~~~~
%Error: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_cfg.v:38:3: syntax error, unexpected '}'
   38 |   } bp_cce_mem_msg_header_s;                                    
      |   ^
%Error: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_cfg.v:38:5: syntax error, unexpected IDENTIFIER, expecting TYPE-IDENTIFIER
   38 |     bp_cce_mem_msg_header_s                      header;        
      |     ^~~~~~~~~~~~~~~~~~~~~~~
%Error: Internal Error: /home/martin/coding/litex/pythondata-cpu-blackparrot/pythondata_cpu_blackparrot/system_verilog/bp_top/src/v/bp_cfg.v:2:8: ../V3ParseSym.h:119: Symbols suggest ending STRUCTDTYPE but parser thinks ending MODULE 'bp_cfg'
    2 | module bp_cfg
      |        ^~~~~~
scanakci commented 3 years ago

Unfortunately, pythondata-cpu-blackparrot is using an outdated version of BlackParrot. I mentioned this issue several times and wanted it to be using a newer version but it is still not the case.

BlackParrot version that I tested with LiteX is 7eb1037637d8515a259e204117b7b1273b1c2941. I can simulate Litex BIOS properly with this BP version. If you replace the system_verilog folder in pythondata-cpu-blackparrot that uses that specific BP commit, you should be able to simulate BIOS. Please let me know if that is not the case. I will also ask again if we can update pythondata-cpu-blackparrot.

The latest litex version is not working properly with BP on FPGA. It is failing at memory tests on FGPA although it is not the case on the simulation level. In the following link, I put an old version of LiteX and BP that I could boot up Linux on FPGA.

https://github.com/black-parrot-hdk/litex/tree/working_linux

Best

troibe commented 3 years ago

Thanks for the steps. I think this is the step I was missing:

If you replace the system_verilog folder in pythondata-cpu-blackparrot that uses that specific BP commit, you should be able to simulate BIOS.

I will try it and report back.

troibe commented 3 years ago

So I finally managed to get the simulation started. Thanks for the help! Copying the system_verilog folder was exactly the step that I was missing.

(The steps mostly align with the steps described in this repo https://github.com/black-parrot-hdk/litex/tree/working_linux)

Just for reproducibility these were the complete steps I took:

  1. Checkout https://github.com/black-parrot-hdk/litex/tree/working_linux (current commit https://github.com/black-parrot-hdk/litex/commit/b79f138908a023a16a446b1fe35ded54ac59e1ba)
  2. Run the setup steps (the apply patches step is the most important one if you already if litex installed) 
    git clone https://github.com/black-parrot/litex
cd litex
./litex_setup.py init dev install --user
./apply_patches.sh
  3. https://github.com/litex-hub/pythondata-cpu-blackparrot/tree/master should be already checked out from the setup (current commit https://github.com/litex-hub/pythondata-cpu-blackparrot/commit/4264d9b0ee43dbb04a94260a6cf9063202996537)
  4. Checkout https://github.com/black-parrot/black-parrot/commit/7eb1037637d8515a259e204117b7b1273b1c2941
  5. Make sure external/basejump_stl is properly checked out to https://github.com/bespoke-silicon-group/basejump_stl/tree/0292a023cdd10ba1e23f4e4cf726dc657066006a like it should be - this caused problems for me because the original git submodule was named https://github.com/black-parrot/basejump_stl/tree/0292a023cdd10ba1e23f4e4cf726dc657066006a - therefore adjust the submodule from black-parrot/basejump_stl to bespoke-silicon-group/basejump_stl!
  6. Really important: copy content from black-parrot to pythondata-cpu-blackparrot/pythondata-cpu-blackparrot/system_verilog overwriting existing files
  7. Checkout https://github.com/scanakci/linux-on-litex-blackparrot (current commit https://github.com/scanakci/linux-on-litex-blackparrot/commit/a4c4458c7d735adeaac70958a19784ef63dcc6fc)
  8. Modify $LITEX/litex/litex_sim.py by replacing soc.add_constant("ROM_BOOT_ADDRESS", 0x40000000) with soc.add_constant("ROM_BOOT_ADDRESS", 0x80000000)
  9. Run litex_sim --cpu-type blackparrot --cpu-variant standard --integrated-rom-size 40960 --output-dir $PWD/build/BP_linux_simu/ --ram-init prebuilt/simulation/Genesys2/boot.bin.uart.simu

  10. The simulation should start. After some waiting you should get this ouput:

    
    __   _ __      _  __
   / /  (_) /____ | |/_/
  / /__/ / __/ -_)>  <
 /____/_/\__/\__/_/|_|
Build your hardware, easily!

(c) Copyright 2012-2020 Enjoy-Digital
(c) Copyright 2007-2015 M-Labs

BIOS built on Apr  3 2021 23:50:53
BIOS CRC passed (8916db0c)

Migen git sha1: 3ffd64c
LiteX git sha1: 4cef798c

--=============== SoC ==================-- CPU: BlackParrotRV64[ima] @ 1MHz ROM: 40KB SRAM: 4KB MAIN-RAM: 262144KB

--============== Boot ==================-- Booting from serial... Press Q or ESC to abort boot completely. sL5DdSMmkekro Timeout Executing booted program at 0x80000000

--============= Liftoff! ===============-- bbl loader ... ... ...



Now I'm quite curious how the simulation performance was for you.
Right now I spent about an hour on the linux boot process.
scanakci commented 3 years ago

Thanks for the details! Steps 3-4-5-6 will vanish once pythondata-cpu-blackparrot repo contains the compatible blackparrot repo with proper submodule external/basejump_stl. I hope that it will get fixed during next week. Boot-up takes roughly 7-8 hours. It may change depend on the host machine you are using.

scanakci commented 3 years ago

With the following commit, LiteX BIOS works fine without replacing systemverilog folder.

https://github.com/litex-hub/pythondata-cpu-blackparrot/commit/0ea975e2b9fb778f9f681eb349f66646bbe82bc6

Closing the issue, let me know if this causes any problem on your end.