Can't boot linux with MAIN_RAM sizes above 512mb

[JoyBed]

I stumbled upon an interesting bug. Linux is unable to boot if the memory size is above 512mb. In litex BIOS the whole ram passes the tests and its working but as linux start to boot it cant boot if I have ram size specified above 512mb. Where does this limitation come from?

[JoyBed]

I dont have these in my local so I dont need to worry about them. My local s like a month old.

[Dolu1990]

I updated the vexii with a fix. Now it work with upstream litex.

[JoyBed]

I updated the vexii with a fix. Now it work with upstream litex.

Funny, for some reason I cant checkout your PR. Nevermind tho, I will manually add the needed files.

[JoyBed]

Also a quick question, does VexiiRiscv keep the offset when talking thru memory bus or not?

[Dolu1990]

@JoyBed Ahhh right, Vexii was using the old code which was removing the offset. I just pushed a fix in this PR. should be good now.

[JoyBed]

@JoyBed Ahhh right, Vexii was using the old code which was removing the offset.

I just pushed a fix in this PR. should be good now.

Either way, with VexiiRiscv I have the same problem as with NaxRiscv. Lockup when DRAM is connected thru memory bus but thru peripheral bus it works.

[Dolu1990]

It lockup only of you try to use more than 512 MB of ram right ?

[JoyBed]

It lockup only of you try to use more than 512 MB of ram right ?

No. At any amount of DRAM. I tried from 32 megs all they way up to 1gig.

[JoyBed]

Now when I was checking it out, why the args for FPU and RVC are commented out? I was wondering why linux are not booting. Also aint it done like in the old VexRiscv where FPU is configurable to be shared between cores?

[Dolu1990]

Now when I was checking it out, why the args for FPU and RVC are commented out?

Do you mean things around : https://github.com/SpinalHDL/VexiiRiscv/blob/4d2ff4b29d04bf033239ace06fb0f61b3600362d/src/main/scala/vexiiriscv/Param.scala#L163 ?

It is only for debug, and isn't enabled for litex. See

  //  Debug modifiers
  val debugParam = sys.env.getOrElse("VEXIIRISCV_DEBUG_PARAM", "0").toInt.toBoolean
  if(debugParam) {

As long as you feed litex with :

python3 -m litex_boards.targets.digilent_nexys_video --cpu-type=vexiiriscv  --with-jtag-tap  --bus-standard axi-lite --vexii-args=" \
--allow-bypass-from=0 --debug-privileged --with-mul --with-div --div-ipc --with-rva --with-supervisor --performance-counters 9 \
--regfile-async --xlen=64 --with-rvc --with-rvf --with-rvd --fma-reduced-accuracy \
--fetch-l1 --fetch-l1-ways=4 --fetch-l1-mem-data-width-min=64 \
--lsu-l1 --lsu-l1-ways=4  --lsu-l1-mem-data-width-min=64 --lsu-l1-store-buffer-ops=32 --lsu-l1-refill-count 2 --lsu-l1-writeback-count 2 --lsu-l1-store-buffer-slots=2  --with-lsu-bypass \
--with-btb --with-ras --with-gshare --relaxed-branch"  --cpu-count=4 --with-jtag-tap  --with-video-framebuffer --with-sdcard --with-ethernet --with-coherent-dma --l2-byte=262144 --update-repo=no  --sys-clk-freq 100000000  --build

It should be debian ready

Also aint it done like in the old VexRiscv where FPU is configurable to be shared between cores?

Right, it isn't. Currently, the FPU is thigtly integrated into the pipeline via some plugins : https://github.com/SpinalHDL/VexiiRiscv/blob/4d2ff4b29d04bf033239ace06fb0f61b3600362d/src/main/scala/vexiiriscv/Param.scala#L660

It would be great to have an lighter alternative (with less performance). I was thinking that maybe a FSM like FPU (instead of fully pipelined) would allow reuse a lot of hardware. Or FPU sharing like in VexRiscv, yes. I implemented things like they are now as i wanted to aim at full perf thigtly coupled FPU, which would also work great in ASIC.

Through peripheral memory, did you got linux to work ?

On my side, quad core debian is very very stable, did a lot of test with it. Also, things like USB host / bluetooth / sdcard / ethernet are working well.

[JoyBed]

Now when I was checking it out, why the args for FPU and RVC are commented out?

Do you mean things around :

https://github.com/SpinalHDL/VexiiRiscv/blob/4d2ff4b29d04bf033239ace06fb0f61b3600362d/src/main/scala/vexiiriscv/Param.scala#L163 ?

It is only for debug, and isn't enabled for litex.

See

  //  Debug modifiers

  val debugParam = sys.env.getOrElse("VEXIIRISCV_DEBUG_PARAM", "0").toInt.toBoolean

  if(debugParam) {

As long as you feed litex with :

python3 -m litex_boards.targets.digilent_nexys_video --cpu-type=vexiiriscv  --with-jtag-tap  --bus-standard axi-lite --vexii-args=" \

--allow-bypass-from=0 --debug-privileged --with-mul --with-div --div-ipc --with-rva --with-supervisor --performance-counters 9 \

--regfile-async --xlen=64 --with-rvc --with-rvf --with-rvd --fma-reduced-accuracy \

--fetch-l1 --fetch-l1-ways=4 --fetch-l1-mem-data-width-min=64 \

--lsu-l1 --lsu-l1-ways=4  --lsu-l1-mem-data-width-min=64 --lsu-l1-store-buffer-ops=32 --lsu-l1-refill-count 2 --lsu-l1-writeback-count 2 --lsu-l1-store-buffer-slots=2  --with-lsu-bypass \

--with-btb --with-ras --with-gshare --relaxed-branch"  --cpu-count=4 --with-jtag-tap  --with-video-framebuffer --with-sdcard --with-ethernet --with-coherent-dma --l2-byte=262144 --update-repo=no  --sys-clk-freq 100000000  --build

It should be debian ready

Also aint it done like in the old VexRiscv where FPU is configurable to be shared between cores?

Right, it isn't.

Currently, the FPU is thigtly integrated into the pipeline via some plugins :

https://github.com/SpinalHDL/VexiiRiscv/blob/4d2ff4b29d04bf033239ace06fb0f61b3600362d/src/main/scala/vexiiriscv/Param.scala#L660

It would be great to have an lighter alternative (with less performance).

I was thinking that maybe a FSM like FPU (instead of fully pipelined) would allow reuse a lot of hardware.

Or FPU sharing like in VexRiscv, yes.

I implemented things like they are now as i wanted to aim at full perf thigtly coupled FPU, which would also work great in ASIC.

Through peripheral memory, did you got linux to work ?

On my side, quad core debian is very very stable, did a lot of test with it. Also, things like USB host / bluetooth / sdcard / ethernet are working well.

Yes, thru peripheral bus its working. But the arguments --with-rvc --with-with-rvf --with-rvd gave error as not recognised so I checked the core.py and they are commented out. Also some lines werent in right order in the core.py file so i reorganised it a bit. When I get home I will send you the core.py I modified from your original one in the PR.

[Dolu1990]

--with-rvc --with-with-rvf --with-rvd

Not reconized by the python litex itself ? or SpinalHDL generation ? note that --with-rvc --with-with-rvf --with-rvd are part of the --vexii-args, they aren't directly feed to litex itself --other-args --vexii-args=" ... --with-rvc --with-with-rvf --with-rvd ... " --other-args

[JoyBed]

--with-rvc --with-with-rvf --with-rvd

Not reconized by the python litex itself ? or SpinalHDL generation ? note that --with-rvc --with-with-rvf --with-rvd are part of the --vexii-args, they aren't directly feed to litex itself --other-args --vexii-args=" ... --with-rvc --with-with-rvf --with-rvd ... " --other-args

Yes, they are not recognised by the SpinalHDL.

EDIT: Nevermind, I see the error. I revised the core.py a bit so no need to call NaxRiscv for repo update. Wanna see it ?

[Dolu1990]

Ahhh i think you have a old version of VexiiRiscv then. I gived you the command with --update-repo=no, i didnt noticed it. --update-repo=recommended

That may explain a lot.

[Dolu1990]

Then the pythondata-cpu-vexiiriscv/pythondata_cpu_vexiiriscv/verilog/ext/vexiiriscv should be on : "fpu_internal", "8a239d10" (after running the soc generation)

[JoyBed]

Yes, but with the core.py version in the PR the update_repo doesnt work, I fixed it and also removed the dependancy on NaxRiscv in the process. Yes, now the pythondata are on "fpu_internal". Can we communicate on some other platform so we will get things done faster so this issue can be closed faster ?

[Dolu1990]

Sure, here is my discord : dolu1990 Would that work for you ?

[JoyBed]

Sure, here is my discord : dolu1990 Would that work for you ?

Yup, I sent you a friend request.

[enjoy-digital]

Hi @Dolu1990, @JoyBed,

have to been able to fix/understand the issue/limitation when discussing directly?

[JoyBed]

Hi @enjoy-digital ! Actually we pinned down what is the problem. It not only affects NaxRiscv's mbus but also other CPUs memory buses, its not a problem of the softcores tho. Problem is tgat Zynq7000 has older AXI3 ports while basically everything here is AXI4. The problem is the lack of WID signal from AXI4 master while AXI3 has it and the PS7 block locks up when receiving AWID of any other value than 0 as the WID on the PS7 is unconnected so interprets WID as 0. We are trying to make a bridge between AXI4 master and AXI3 slave. Only unaffected softcore is Rocket as its TileLink to AXI4 bridge uses AWID = 0 so lack of WID is not a problem.

[enjoy-digital]

Thanks @JoyBed for the feedback. Regarding your application, do you think improvement should be made to LiteX to at least maybe prevent things from building? If you could provide more information about the zynq7000 integration you are doing an minimal repro, we could try to raise an error if this case is not supposed to be supported and shouldn't build.