enjoy-digital / litex_vexriscv_smp_test

VexRiscv-SMP integration test with LiteX.
24 stars 6 forks source link

Core count greater than 2 broken #2

Closed mtdudek closed 4 years ago

mtdudek commented 4 years ago

Recently I tried running vexriscv SMP with 4 cores on arty a7 35t. But I was greeted with failed memtest. I did git bisect to narrow down which commit has broken 4 core implementation, and unfortunately I found out it is this one 43aed3a. Here is log I got when I was running newest version

[bus: 0x0]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x1]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x2]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x3]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x7]: 0x00090700 vs 0xaaaaaaaa
[bus: 0xb]: 0x00000007 vs 0xaaaaaaaa
[bus: 0xf]: 0x0000000b vs 0xaaaaaaaa
[bus: 0x10]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x11]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x12]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x13]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x17]: 0x55555555 vs 0xaaaaaaaa
[bus: 0x1b]: 0x00000017 vs 0xaaaaaaaa
[bus: 0x1f]: 0x0000001a vs 0xaaaaaaaa
[bus: 0x20]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x21]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x22]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x23]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x27]: 0x55555555 vs 0xaaaaaaaa
[bus: 0x2b]: 0x55555555 vs 0xaaaaaaaa
[bus: 0x30]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x31]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x32]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x33]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x37]: 0x00000033 vs 0xaaaaaaaa
[bus: 0x3f]: 0x0000003b vs 0xaaaaaaaa
[bus: 0x40]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x41]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x42]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x43]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x47]: 0x00000041 vs 0xaaaaaaaa
[bus: 0x4b]: 0x009e002f vs 0xaaaaaaaa
[bus: 0x50]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x51]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x52]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x53]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x57]: 0xaa00aa53 vs 0xaaaaaaaa
[bus: 0x5b]: 0x00000057 vs 0xaaaaaaaa
[bus: 0x5f]: 0x0000005b vs 0xaaaaaaaa
[bus: 0x60]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x61]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x62]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x63]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x67]: 0x55555555 vs 0xaaaaaaaa
[bus: 0x6b]: 0x00000066 vs 0xaaaaaaaa
[bus: 0x6f]: 0x287e5038 vs 0xaaaaaaaa
[bus: 0x70]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x71]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x72]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x73]: 0x00000000 vs 0xaaaaaaaa
[bus: 0x77]: 0x00000072 vs 0xaaaaaaaa
[bus: 0x7f]: 0x55555555 vs 0xaaaaaaaa
[bus 0x0]: 0x00000000 vs 0x55555555
[bus 0x1]: 0x00000000 vs 0x55555555
[bus 0x2]: 0x00000000 vs 0x55555555
[bus 0x3]: 0x00000000 vs 0x55555555
[bus 0x7]: 0x00090700 vs 0x55555555
[bus 0xb]: 0x00000007 vs 0x55555555
[bus 0xf]: 0x0000000b vs 0x55555555
[bus 0x10]: 0x00000000 vs 0x55555555
[bus 0x11]: 0x00000000 vs 0x55555555
[bus 0x12]: 0x00000000 vs 0x55555555
[bus 0x13]: 0x00000000 vs 0x55555555
[bus 0x1b]: 0x00000017 vs 0x55555555
[bus 0x1f]: 0x0000001a vs 0x55555555
[bus 0x20]: 0x00000000 vs 0x55555555
[bus 0x21]: 0x00000000 vs 0x55555555
[bus 0x22]: 0x00000000 vs 0x55555555
[bus 0x23]: 0x00000000 vs 0x55555555
[bus 0x2f]: 0xaaaaaaaa vs 0x55555555
[bus 0x30]: 0x00000000 vs 0x55555555
[bus 0x31]: 0x00000000 vs 0x55555555
[bus 0x32]: 0x00000000 vs 0x55555555
[bus 0x33]: 0x00000000 vs 0x55555555
[bus 0x37]: 0x00000033 vs 0x55555555
[bus 0x3b]: 0xaaaaaaaa vs 0x55555555
[bus 0x3f]: 0x0000003b vs 0x55555555
[bus 0x40]: 0x00000000 vs 0x55555555
[bus 0x41]: 0x00000000 vs 0x55555555
[bus 0x42]: 0x00000000 vs 0x55555555
[bus 0x43]: 0x00000000 vs 0x55555555
[bus 0x47]: 0x00000041 vs 0x55555555
[bus 0x4f]: 0x55aa55aa vs 0x55555555
[bus 0x50]: 0x00000000 vs 0x55555555
[bus 0x51]: 0x00000000 vs 0x55555555
[bus 0x52]: 0x00000000 vs 0x55555555
[bus 0x53]: 0x00000000 vs 0x55555555
[bus 0x57]: 0xaa00aa53 vs 0x55555555
[bus 0x5b]: 0x00000057 vs 0x55555555
[bus 0x5f]: 0x0000005b vs 0x55555555
[bus 0x60]: 0x00000000 vs 0x55555555
[bus 0x61]: 0x00000000 vs 0x55555555
[bus 0x62]: 0x00000000 vs 0x55555555
[bus 0x63]: 0x00000000 vs 0x55555555
[bus 0x6b]: 0x00000066 vs 0x55555555
[bus 0x6f]: 0x287e5038 vs 0x55555555
[bus 0x70]: 0x00000000 vs 0x55555555
[bus 0x71]: 0x00000000 vs 0x55555555
[bus 0x72]: 0x00000000 vs 0x55555555
[bus 0x73]: 0x00000000 vs 0x55555555
[bus 0x77]: 0x00000072 vs 0x55555555
[bus 0x7b]: 0xaaaaaaaa vs 0x55555555
[########################################]
[########################################]
- bus errors:  102/256
- addr errors: 8191/8192
- data errors: 524288/524288
Memtest KO
Memory initialization failed

Number of bus errors was different between runs, as was addr errors in newest version. On the other hand on commit 43aed3a I always got 128 errors.

Dolu1990 commented 4 years ago

Hi,

Which board are you using ? Else, we are currently merging the SMP stuff into litex, should be done this week. I that litex merge, 4 cores worked fine for (Arty35T) I will notify you as soon this is released (very very soon) :)

Dolu1990 commented 4 years ago

I tested quad core with the branch "integration", it worked fine on arty. If you want to try it you will need to update litex, and find the prebuild software binaries here : https://github.com/Dolu1990/linux-on-litex-vexriscv-prebuilt.git

I loaded it using uart.

If you change the config, you might need to install java and SBT to enable the automatic generation of the cluster which fit the parameters.

Dolu1990 commented 4 years ago

Should probably go in jail :


./make.py --board=arty --cpu-count=3 --build --load

root@buildroot:~# cat /proc/cpuinfo 
processor   : 0
hart        : 0
isa     : rv32ima
mmu     : sv32
uarch       : spinalhdl,vexriscv

processor   : 1
hart        : 1
isa     : rv32ima
mmu     : sv32
uarch       : spinalhdl,vexriscv

processor   : 2
hart        : 2
isa     : rv32ima
mmu     : sv32
uarch       : spinalhdl,vexriscv
...
mtdudek commented 4 years ago

Today I was able to try again, and now everything works.