search

enjoy-digital / litex

Build your hardware, easily!
Other
2.9k stars 556 forks source link

FemtoRV: Finish integration. #1096

Closed enjoy-digital closed 2 years ago

enjoy-digital commented 2 years ago

The upstream versions does not compile correctly with Verilator with ADDR_WIDTH set to 32. So the following patched version is currently used: femtorv32_quark.v.txt.

BrunoLevy commented 2 years ago

Hi, I have just pushed a new version of femtorv32_quark that will not have the problem (removed ADDR_PAD and added lint_off directives).

BrunoLevy commented 2 years ago

P.S. If you are interested, besides the tiny "quark" version, there are other versions of femtorv (the biggest one, femtorv-petitbateau supports RV32F).

enjoy-digital commented 2 years ago

Hi @BrunoLevy,

great, thanks for sharing this. (Sorry I wanted to report this but just used this patched version as a quick workaround for now). I'll switch the repo to your new version then :) (and am definitely interested to add support for the other variants).

There is also another issue on hardware when testing the SDRAM, but this could be related to my understanding of the memory interface. Do you have some documentation of the implemented interface?

BrunoLevy commented 2 years ago

OK, I'll try to run it in LiteX and see what happens. I will be very happy to contribute (I just need to find some time to learn how to use LiteX, then I'll come back to you).

BrunoLevy commented 2 years ago

Here is how femtorv32 memory interface works:

Writing:

cycle 1: mem_wmask goes non-zero (and indicates which byte should be written) mem_address mem_wdata cycle 2...n: mem_wbusy goes high and stays high until writing operation is completed (it can be kept low if writing is done in 1 cycle)

Reading:

cycle 1: mem_rstrb goes high mem_address cycles 2...n mem_rbusy goes high and stays high until memory read is completed once mem_rbusy is low, data is available on mem_rdata

BrunoLevy commented 2 years ago

I've just installed LiteX and lxsim works on my box. How can I get the version that has femtorv ? (is it in a separate branch), or is there a tutorial to add a CPU ? Thanks in advance, -- B

enjoy-digital commented 2 years ago

Hi @BrunoLevy,

great, thanks for the information. Also happy to work together on the FemtoRV integration and discover the other variants!

If litex_sim already works on your box, switching to femtorv should only be a matter of adding --cpu-type=femtorv :)

If you want to reproduce the issue with SDRAM test, it's possible to build the design on an Arty (which I think you have): python3 -m litex_boards.targets.digilent_arty --cpu-type=femtorv --build --load

I'll check the FSM I implemented to convert the memory bus to wishbone to see if it matches your explanation.

Florent

BrunoLevy commented 2 years ago

Many thanks !

Note: I have not installed Vivado (I'm using symbiflow that does not seem to be supported yet by LiteX), but I also have an ULX3S that has SDRAM (and I could build for it using yosys/nextpnr-treillis).

Many thanks in advance, Best, -- B

enjoy-digital commented 2 years ago

@BrunoLevy: The patched version is now avoided with https://github.com/enjoy-digital/litex/commit/605b6aff7c98ebbcbdada8a795f32558c955d437, thanks for the update on this.

I'm going to see if I also reproduce the issue on the ULX3S and will share the command if so.

enjoy-digital commented 2 years ago

I can reproduce the issue on the ULX3S with the following command: python3 -m litex_boards.targets.radiona_ulx3s --cpu-type=femtorv --build --load

        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2021 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS built on Nov 19 2021 12:29:16
 BIOS CRC passed (935d8489)

 Migen git sha1: 9a0be7a
 LiteX git sha1: 605b6aff

--=============== SoC ==================--
CPU:        FemtoRV @ 50MHz
BUS:        WISHBONE 32-bit @ 4GiB
CSR:        32-bit data
ROM:        128KiB
SRAM:       8KiB
L2:     8KiB
SDRAM:      32768KiB 16-bit @ 50MT/s (CL-2 CWL-2)

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Switching SDRAM to hardware control.
Memtest at 0x40000000 (2.0MiB)...
  Write: 0x40000000-0x40200000 2.0MiB     
   Read: 0x40000000-0x40200000 2.0MiB     
  bus errors:  256/256
  addr errors: 8189/8192
  data errors: 524288/524288
Memtest KO
Memory initialization failed

--============= Console ================--

litex>
enjoy-digital commented 2 years ago

In fact we can directly reproduce the issue in simulation when enabling the SDRAM model:

litex_sim --cpu-type=femtorv --with-sdram

        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2021 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS built on Nov 19 2021 12:37:20
 BIOS CRC passed (bf7d7c6e)

 Migen git sha1: 9a0be7a
 LiteX git sha1: 605b6aff

--=============== SoC ==================--
CPU:        FemtoRV @ 1MHz
BUS:        WISHBONE 32-bit @ 4GiB
CSR:        32-bit data
ROM:        128KiB
SRAM:       8KiB
L2:     8KiB
SDRAM:      65536KiB 32-bit @ 1MT/s (CL-2 CWL-2)

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Switching SDRAM to hardware control.
Memtest at 0x40000000 (8.0KiB)...
  Write: 0x40000000-0x40002000 8.0KiB   
   Read: 0x40000000-0x40002000 8.0KiB   
  bus errors:  256/256
  addr errors: 2047/2048
  data errors: 2048/2048
Memtest KO
Memory initialization failed

--============= Console ================--

litex>

With another CPU (VexRiscv for example): litex_sim --cpu-type=vexriscv --with-sdram

        __   _ __      _  __
       / /  (_) /____ | |/_/
      / /__/ / __/ -_)>  <
     /____/_/\__/\__/_/|_|
   Build your hardware, easily!

 (c) Copyright 2012-2021 Enjoy-Digital
 (c) Copyright 2007-2015 M-Labs

 BIOS built on Nov 19 2021 12:38:19
 BIOS CRC passed (9236a67c)

 Migen git sha1: 9a0be7a
 LiteX git sha1: 605b6aff

--=============== SoC ==================--
CPU:        VexRiscv @ 1MHz
BUS:        WISHBONE 32-bit @ 4GiB
CSR:        32-bit data
ROM:        128KiB
SRAM:       8KiB
L2:     8KiB
SDRAM:      65536KiB 32-bit @ 1MT/s (CL-2 CWL-2)

--========== Initialization ============--
Initializing SDRAM @0x40000000...
Switching SDRAM to software control.
Switching SDRAM to hardware control.
Memtest at 0x40000000 (8.0KiB)...
  Write: 0x40000000-0x40002000 8.0KiB   
   Read: 0x40000000-0x40002000 8.0KiB   
Memtest OK
Memspeed at 0x40000000 (Sequential, 8.0KiB)...
  Write speed: 354.2KiB/s
   Read speed: 276.6KiB/s

--============== Boot ==================--
Booting from serial...
Press Q or ESC to abort boot completely.
sL5DdSMmkekro
Timeout
No boot medium found

--============= Console ================--

litex>

This will simplify the debug (the traces can be enabled by adding --trace to the command and will then be located at build/sim/gateware/sim.vcd)

I'll try to have a closer look with your description of the memory interface.

enjoy-digital commented 2 years ago

@BrunoLevy: Thanks for the clarifications on FemtoRV's bus, the fact that the address/data/mask values are not maintained after the first cycle was not handled correctly. This is now fixed with https://github.com/enjoy-digital/litex/commit/54a4e6c1f626ffe86aabfb46c447b1afe1641440, see: https://twitter.com/enjoy_digital/status/1461709551334567942 :)

Now we'll just have to support the other variants!

enjoy-digital commented 2 years ago

https://github.com/enjoy-digital/litex/commit/3c3884b1ea545905b9cf20cd2682400f95f1bcd3 adds initial support for the variant (with Quark & Tachyon for now).

tcal-x commented 2 years ago

Note: I have not installed Vivado (I'm using symbiflow that does not seem to be supported yet by LiteX), but I also have an ULX3S that has SDRAM (and I could build for it using yosys/nextpnr-treillis).

@BrunoLevy -- I'd like to clarify that LiteX does support SymbiFlow (although the user is responsible for acquiring SymbiFlow and making sure it's in their path, I usually use my symbiflow-examples repo) -- use --toolchain=symbiflow with XC7 targets.

enjoy-digital commented 2 years ago

@BrunoLevy: https://github.com/enjoy-digital/litex/commit/10155c7d6fce8841e4c39283f7e27b722c0579a0 adds the initial support for the other variants. I also listed the list of issues: I think electron/intermissum/gracilis would just with the the same changes on ADDR_PAD you just did on quark and electron. I could have a closer look at petitbateau once the other variant will be working.

BTW: To select a variant in simulation or on the target, you can just add --cpu-variant=quark, --cpu-variant=electron, etc...

BrunoLevy commented 2 years ago

Wonderful ! I have fixed the variants (would like to be able to say the same about COVID ;-) and pushed a new version. It is absolutely great that it direcly wgets the verilog from github, love it !

I tested the largest one (petitbateau with RV32F), on my ULX3S, it validates only at 30 MHz (I need to play with the flags, probably better with ABC9 / nowideluts). It seems to work fine. I had to do a couple of fixes in litex/litex/soc/cores/cpu/femtorv/core.py: Line 30: "intermissum" : "femtorv32_intermissum" (cut-and-paste typo to be fixed) Line 51: ABI flags for RV32F need to be changed to ilp32f : "-march=rv32imcf -mabi=ilp32f"

I have a couple of quick questions:

tcal-x commented 2 years ago

Hi @BrunoLevy , have you seen the small demo software? https://twitter.com/enjoy_digital/status/1341095343816118272

It shows how lxterm --kernel <software.bin> --speed 115200 /dev/ttyUSB0 (substitute the actual tty!) can be used to load a binary into RAM.

enjoy-digital commented 2 years ago

Great, that was fast @BrunoLevy :) LiteX typos have been fixed with https://github.com/enjoy-digital/litex/commit/e383212ffbe47cfcb01ffe9547e4d5d6345e3800 and I can confirm the variants are working fine in simulation. (There seems to have an issue in sim with petitbateau, but I'll look at it another day).

For the framebuffer, when built with --with-video-framebuffer the Framebuffer should be automatically enabled by the BIOS: https://github.com/enjoy-digital/litex/blob/master/litex/soc/software/bios/main.c#L171-L177, but does not do anything with it, so it should just output SDRAM contents. (We only use it with booted software).

BrunoLevy commented 2 years ago
BrunoLevy commented 2 years ago

Tried today on the ULX3S picorv32 + SDRAM + video-framebuffer: same behavior (seems that no video signal is generated on the HDMI, and sdram test is KO).

enjoy-digital commented 2 years ago

Thanks for the updates @BrunoLevy, I'll be able to do a test on ULX3S on next monday.

BrunoLevy commented 2 years ago

litex-boards/litex_boards/targets/radiona_ulx3s.py line 129: by commenting-out the line that says FIXME sdram test is OK (but it seems it still does not generate a video signal)

BrunoLevy commented 2 years ago

Hi, I have noticed that compilation flags do not change when femtorv32-variant is specified, I think that a couple of things need to be fixed in femtorv/core.py:

With these changes, it seems that it generates the BIOS with the correct flags (they are correct in build/radiona_ulx3s/software/include/generated/variables.mak)

However, litex_bare_metal_demo does not work (it compiles with ilp32f ABI as required, but tries to link a library compiled with ilp32 ABI)

enjoy-digital commented 2 years ago

Thanks @BrunoLevy, the first issues should be fixed with:

I'm going to do a test with Video on the ULX3S.

enjoy-digital commented 2 years ago

@BrunoLevy: The framebuffer seems to be working correctly here on the ULX3S (just output SDRAM contents): DSC_0588

But I'm testing with an ULX3S v1.7 (early version) and I'm wondering if the GPDI1 has been reworked on version from the campaign and would then need to be adapted.

enjoy-digital commented 2 years ago

The pins constraints seem similar on the GPDI for the two revisions. You could also eventually try to use different timings (you can find some here: https://github.com/enjoy-digital/litex/blob/master/litex/soc/cores/video.py#L24) and also adapt the hdmi/hdmi5x clock or try with another monitor.

BrunoLevy commented 2 years ago

Thanks ! I'll try to tinker with it until I get an image :-) In case it helps, my simple HDMI design for the ULX3S: https://github.com/BrunoLevy/learn-fpga/tree/master/Basic/ULX3S/ULX3S_hdmi

BrunoLevy commented 2 years ago

I confirm that all the other issues are fixed in the update: the bare metal demo compiles in RV32IMFC mode. Once I manage to make the HDMI work (will try more this WE), I'll write a small raytracing demo.

enjoy-digital commented 2 years ago

Thanks, I just removed the debug line that was causing you an issue with https://github.com/litex-hub/litex-boards/commit/70c0dbb18525d5039f467bc4fc8c68b8a34a7047. I'll have a closer look at your code and will also test it on my setup.

BrunoLevy commented 2 years ago

Hi @enjoy-digital, here are the results of today's test:

BrunoLevy commented 2 years ago

Hi, tested today software generation with petitbateau variant. BIOS compiles OK, but baremetal example has linking problems:

blevy@redshift:~/Programming/LiteX$ litex_bare_metal_demo --build-path build/radiona_ulx3s/
cp: -r not specified; omitting directory '/home/blevy/Programming/LiteX/litex/litex/soc/software/demo/__pycache__'
 CC       isr.o
 CC       donut.o
 CC       helloc.o
 CC       main.o
 CC       demo.elf
/usr/lib/riscv64-unknown-elf/bin/ld: crt0.o: can't link soft-float modules with single-float modules
/usr/lib/riscv64-unknown-elf/bin/ld: failed to merge target specific data of file crt0.o
collect2: error: ld returned 1 exit status
make: *** [Makefile:21: demo.elf] Error 1
BrunoLevy commented 2 years ago

Finally managed to create a baremetal firmware with tinyraytracer and OLED display:

crt0.o: $(CPU_DIRECTORY)/crt0.S $(BUILD_DIR)/software/include/generated/variables.mak
        $(assemble)

Best wishes, -- Bruno

BrunoLevy commented 2 years ago

(moved here: https://github.com/BrunoLevy/learn-fpga/tree/master/FemtoRV/FIRMWARE/LiteX/DemoBundle)

BrunoLevy commented 2 years ago

Current status:

enjoy-digital commented 2 years ago

Thanks @BrunoLevy, https://github.com/litex-hub/litex-boards/commit/b5008a2d5c2876046d0ecb77fa2d307169ad6783 should fix the video issue (no need to specify--revision=2.0 for you since2.0 is already the default).

I'll try to reproduce the SDRAM issue.

BrunoLevy commented 2 years ago
BrunoLevy commented 2 years ago

Project status: reached milestone #1 (raytracer running in 640x480 with framebuffer, Yeehaa !!)

Summary of the list of changes I made to make it work :

Remaining issues:

Suggestions for new functionalities:

I have that in the 'Femto Graphic Adapter' (see link below), it makes it easy to port old 90-ish demos, this probably includes running Doom (this one may be important from a 'marketing' point of view :-): https://github.com/BrunoLevy/learn-fpga/blob/master/FemtoRV/RTL/DEVICES/FGA.v

enjoy-digital commented 2 years ago

We can probably close this now since FemtoRV is well integrated. Doing the integration together was very pleasant @BrunoLevy, happy to discuss the other points that could be improved in the future in specific issues.