Open bacintom opened 1 year ago
Hi @bacintom,
thanks a lot for the detailed report. The Tang Primer is currently probably not the best board to start with since all features haven't been validated yes (especially the DDR3, SDCard), but that's really interesting that you got the DDR3 working by adjusting cl/cwl since with this, I think you are the first to get no errors with the memtest: I'll have a closer look, try to understand and do the proper fixes in upstream. (Note: This is also related to https://github.com/enjoy-digital/litex/issues/1649).
I'll also look at the other issues/workaround while looking at this.
Hi @bacintom We have tried your fix for DDR3 with two tang Primer 20k variants:
We have to find why there is a different behaviour between these two chips and how integrate your modification/update (at litedram phy level).
For RAM: size parameters are good and matchs RAM datasheet we have to investigate why a factor of two is introduces when size is computed.
Thanks!
Thanks @trabucayre, @bacintom: Can you tell us which DDR3 variant is mounted on your Primer (SKHynix or ICMax)?
Hi! Thanks for yout replies! In my case, it is a SK Hynix. The mystery thickens. I somehow managed to take a photo with a readable part number, let me attach it here.
@trabucayre to be honest I don't understand much about RAM but here's my take about the size issue:
I found a datasheet of H5TQ1Gx3EFR-xxx modules
In my case the exact number is H5TQ1G63EFR-PBC the "63" (unlike "83") means it is a 64M x 16 configuration
There is a table called "Row and column address table"
The right column should be used -There we have "row address" ranging from A0 to A12, I guess those are pin numbers... That's 13 bits, so the field 'nrows' of the DDR3Module should be 2^13 = 8192 (not 16384 as in the master) -Column address is A0-A9 -> 10bits -> 2^10=1024 OK
What do you think?
Hi @bacintom could you try this (once again, not sure about proper timings): https://github.com/enjoy-digital/litex/issues/1649#issuecomment-1656509253
@e-yes As long as I set nrows=8192 (the quickest way to test is, take an unmodified master and set the value in MT41J128M16) it will report the correct amount 128MiB, no other change needed.
So your change works and reports 128MiB. It will also fail the test in BIOS, and succeed after calling the sdram_mr_write/sdram_cal commands.
Thanks to the fix in from @trabucayre in https://github.com/enjoy-digital/litedram/commit/6dadc11b21a8253aa20305f1a54c1a2f1f4bbfd6 I can make it pass and this configuration for the ddr3 module:
class H5TQ1G63EFR(DDR3Module):
# geometry
nbanks = 8
nrows = 8192
ncols = 1024
# timings
technology_timings = _TechnologyTimings(tREFI=64e6/8192, tWTR=(4, 7.5), tCCD=(4, None), tRRD=(4, 6), tZQCS=(64, 80))
speedgrade_timings = {
"800": _SpeedgradeTimings(tRP=15, tRCD=15, tWR=15, tRFC=(120, None), tFAW=(None, 50), tRAS=37.5),
"1066": _SpeedgradeTimings(tRP=13.125, tRCD=13.125, tWR=15, tRFC=(160, None), tFAW=(None, 50), tRAS=37.5),
"1600": _SpeedgradeTimings(tRP=13.75, tRCD=13.75, tWR=15, tRFC=(160, None), tFAW=(None, 40), tRAS=35),
}
speedgrade_timings["default"] = speedgrade_timings["800"]
Note the nrows as e-yes is also mentioning.
I am not sure how I got to those numbers as such, but some of them are either taken by searching for tRP numbers from other modules and probably https://github.com/enjoy-digital/litex/issues/1649#issuecomment-1656509253 -- but it is what I tested.
I get highest speed with the 800 timing, ... I don't know if that is expected.
Write speed: 23.4MiB/s
Read speed: 8.3MiB/s
For my 1066 and 1600 configration:
Write speed: 22.2MiB/s
Read speed: 7.8MiB/s
@bacintom after I got working sdram, the sdcard also worked for me. At least it would boot the demo.bin from a boot.json.
@nickoe thanks I hadn't noticed something was pushed to litedram master.
With @trabucayre 's commit, the bios passes the sdram check successfully. No need to tweak with mode register commands anymore and that's nice
If I use @nickoe 's timings, sdcard (non-spi) also works. Well, at least I can go as far as spisdcard: it shows "liftoff!" and freezes.
So, issues 1,2,3 are pretty much fixed. Thanks everyone!
@bacintom Regarding the liftoff issue, it can be seen from the following link that linux-on-litex-vexriscv supports vexriscv_smp,so it should be --cpu-type vexriscv_smp, but it will compile and prompt an error. https://github.com/litex-hub/linux-on-litex-vexriscv
I have tried to compile the gateware for vexriscv_smp, but it doesn't fit and the compilation failed.
Hi,
I got over my liftoff issue. I did the following :
-update all litex. Patches about supporting tang primer 20k have been pushed recently.
-check the files in the SD card using md5sum. One of my files had a wrong signature so I had to copy it again. I would recommend to double-check like this: unmount, unplug sd card, plug sd card, remount, verify checksum. This is to avoid being tricked by the OS, it could keep files in cache and skip reading from the actual SD card.
-building with linux-on-litex-vexriscv: ./make.py --board=sipeed_tang_primer_20k --toolchain=gowin --build --load
(a patch added sipeed_tang_primer_20k on August 29th)
With this the board can boot linux!
I could also build with video_terminal, but it's unstable. Only part of the text appears on screen and the board will freeze before linux is fully booted.
@bacintom hi , i firstly use litex to build tang-primer-20k linux , but in this linux , i find commands can not be uesd such as cd ,ls and so on. How do you know about these commands? Is there a corresponding instruction manual?
hi , i use same command:./make.py --board=sipeed_tang_primer_20k --toolchain=gowin --build --load
,but i got
same error like 4 :
how do you solve this problem
@Sujunyou cd and ls work fine in this linux, so I'm not sure what your problem is...
Are you sure you set up a linux?... If you just build litex it will bring you to the litex bios, only a few bios-level commands are available.
There are some explanations here: https://github.com/litex-hub/linux-on-litex-vexriscv in particular, you should download linux_2022_03_23.zip from https://github.com/litex-hub/linux-on-litex-vexriscv/issues/164 , format an SD card with a FAT16 or FAT32 partition and extract the files there. However, about the file rv32.dtb I use the one built by make.py in the linux-on-litex-vexriscv/images because it has a different signature, I'm not sure whether it is important.
@Sujunyou cd and ls work fine in this linux, so I'm not sure what your problem is...
Are you sure you set up a linux?... If you just build litex it will bring you to the litex bios, only a few bios-level commands are available.
There are some explanations here: https://github.com/litex-hub/linux-on-litex-vexriscv in particular, you should download linux_2022_03_23.zip from litex-hub/linux-on-litex-vexriscv#164 , format an SD card with a FAT16 or FAT32 partition and extract the files there. However, about the file rv32.dtb I use the one built by make.py in the linux-on-litex-vexriscv/images because it has a different signature, I'm not sure whether it is important.
thanks fo your reply.i follow your steps to build linux(use command:python3 -m litex_boards.targets.sipeed_tang_primer_20k --with-sdcard --cpu-type vexriscv --cpu-variant linux --with-video-terminal --build --load
)but i got following error:
`ERROR (RP0006) : The number(61480(56055 LUTs, 1537 ALUs, 0 ROM16s, 648 SSRAMs)) of logic in the design exceeds the resource limit(20736) of current device
ERROR (RP0001) : The number(76249) of DFF(DL) in the design exceeds the resource limit(16173) of current device(GW2A-LV18PG256C8/I7)
GowinSynthesis finish
while executing
"run all"
(file "run.tcl" line 11)
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/root/python-litex/litex-boards/litex_boards/targets/sipeed_tang_primer_20k.py", line 239, in
@sususjysjy The issue 4 is not solved, I just disable the check that fails by editing gowin_gw1n.py, it's also fully explained how to do it in my first post...
@sususjysjy I'm confused, I was replying to @Sujunyou there, are you the same person?
I get the same error as you nowadays if I use litex_boards, the error means the design won't fit in the FPGA. But it works if I use linux-on-litex-vexriscv/make.py instead... I don't know why 🤷
@sususjysjy I'm confused, I was replying to @Sujunyou there, are you the same person?
I get the same error as you nowadays if I use litex_boards, the error means the design won't fit in the FPGA. But it works if I use linux-on-litex-vexriscv/make.py instead... I don't know why 🤷
@sususjysjy I'm confused, I was replying to @Sujunyou there, are you the same person?
I get the same error as you nowadays if I use litex_boards, the error means the design won't fit in the FPGA. But it works if I use linux-on-litex-vexriscv/make.py instead... I don't know why 🤷
sorry,these all my account. I find reasons following: in litex,if use above command,generate all the sdcard; but if you use linux-on-litex-vexriscv , it generates spisdcard, it's smaller than litex
@sususjysjy I can build with make.py and enable sdcard instead of spisdcard. It will build successfully. The size difference must be somewhere else?
But sdcard+linux-on-litex-vexriscv doesn't work on master with the tang primer 20k... I've just tried with @nickoe 's RAM timings, but I could not boot, sdcard_init fails again. 🤷🤷🤷
So, I think it's better to use spisdcard for now.
yeah,i use make.py, spi sdcard can be built succeessfully ; but in litex, i use command python3 -m litex_boards.targets.sipeed_tang_primer_20k --with-spi-sdcard --build
, it will show me resources are not available。
i'm so confused for this
@sususjysjy I can build with make.py and enable sdcard instead of spisdcard. It will build successfully. The size difference must be somewhere else?
But sdcard+linux-on-litex-vexriscv doesn't work on master with the tang primer 20k... I've just tried with @nickoe 's RAM timings, but I could not boot, sdcard_init fails again. 🤷🤷🤷
So, I think it's better to use spisdcard for now.
hi , you said "you can use make.py and enable sdcard",in make.py,it builds spisdcard , should i convert spisdcard to sdcard? The following is the default version:
if i convert to sdcard , it will occurr error May I take a look at your project?
@sususjysjy what version of GOWIN IDE do you use?
@sususjysjy what version of GOWIN IDE do you use?
thanks for your reply i use the latest version 1.9.8.11_education now i meet the new problem , i build .fs and load to tang-primer-20k , and bios work fine but when i want to boot linux(provided by (linux-on-litex-vexriscv) from sdcard , terminal shows following error: i don't know what's wrong
You are on the right path I believe. Can you give us more details? I am even jealous, that I didn't come so close to solving this puzzle)
You are on the right path I believe. Can you give us more details? I am even jealous, that I didn't come so close to solving this puzzle)
i use linux-on-litex-vexriscv , use the command ./make.py --board=sipeed_tang_primer_20k --toolchain=gowin --build
,then i load the .fs to board .
and then , i put five files in sdcard ,and boot the board
finally , i see the following terminal:
Ok this is amazing. I had no idea this board had the FPGA resources to boot linux on! Is the vexriscv core the only one to use for this?
I suspect vexriscv_smp is required at least for these prebuilt images.
i have solve this problem.thanks all
It would be nice if you share some details...
It would be nice if you share some details...
ok. In a few days, I will post my strategy for how to solve the problem
I got it working on my board by ONLY replacing the image files with the ones from the download site. If you overwrite the DTB that was created by litex when you build for this board all the hardware will be wrong.
Try without overwriting rv32.dtb and boot.json that the litex-on-linux-rexriscv build creates.
I got it working on my board by ONLY replacing the image files with the ones from the download site. If you overwrite the DTB that was created by litex when you build for this board all the hardware will be wrong.
Try without overwriting rv32.dtb and boot.json that the litex-on-linux-rexriscv build creates.
you are right.:) The same issues occurr in tang_mega_138k. So confused about memory.(memory init fail) I tried all the combinations of cl and cwl , but it works badly. I think this is related to the timing issue caused by Gowin synthesis
Hi,
I got over my liftoff issue. I did the following :
-update all litex. Patches about supporting tang primer 20k have been pushed recently.
-check the files in the SD card using md5sum. One of my files had a wrong signature so I had to copy it again. I would recommend to double-check like this: unmount, unplug sd card, plug sd card, remount, verify checksum. This is to avoid being tricked by the OS, it could keep files in cache and skip reading from the actual SD card.
-building with linux-on-litex-vexriscv:
./make.py --board=sipeed_tang_primer_20k --toolchain=gowin --build --load
(a patch added sipeed_tang_primer_20k on August 29th)With this the board can boot linux!
I could also build with video_terminal, but it's unstable. Only part of the text appears on screen and the board will freeze before linux is fully booted.
Hi @bacintom , I run ./make.py --board=sipeed_tang_primer_20k --toolchain=Gowin --with-video-terminal --build
and got an error:
make.py: error: unrecognized arguments: --with-video-terminal
How can I build the bitstream with video_terminal? Thank you so much!
I find out that the big different between build up the litex over the make.py for the linux-on-litex and the way we build it up is to set the l2 cache if you like to have the same size and be able to build it up without error I could recommend to build up the litex with the below option: python -m litex_boards.targets.sipeed_tang_primer_20k --l2-size 512 --with-spi-sdcard --build --with-video-terminal please update all of your packages with pip before try to run the above command. To let the linux work as you expected the linux kernel also need to have the related driver its not enough to just add the HW like framebuffer and usb to the litex only. The primer 20k is not a big fpga and as far as I find see its not capable of running many peripherals at the same time. You need about 10K to 15K LUT for a soft CPU Around 8k for HDMI UART 1 or 2 K So the capacity is so tide for expansion. A real deal on linux running board might be something like Arty A7 100K you could run all you need and drive HD monitor
Hi, @gr8linux , thank you so much for your suggestions. I use python3 -m litex_boards.targets.sipeed_tang_primer_20k --l2-size 512 --with-spi-sdcard --build --with-video-terminal
to generate bitstream file and load it on my board. Then I use the files of https://github.com/litex-hub/linux-on-litex-vexriscv/issues/164#issue-767463348 and try to boot Linux from SDCard. However, it finally stuck in the "liftoff" phase. Is it a problem with the related driver as you mention above? Is there a way to slove that? Thanks!
Hi, @gr8linux , thank you so much for your suggestions. I use
python3 -m litex_boards.targets.sipeed_tang_primer_20k --l2-size 512 --with-spi-sdcard --build --with-video-terminal
to generate bitstream file and load it on my board. Then I use the files of https://github.com/litex-hub/linux-on-litex-vexriscv/issues/164#issue-767463348 and try to boot Linux from SDCard. However, it finally stuck in the "liftoff" phase. Is it a problem with the related driver as you mention above? Is there a way to slove that? Thanks!
This might be related to the cpu version and LiteDRAM module maybe the below option help:
python -m litex_boards.targets.sipeed_tang_primer_20k --l2-size 512 --build --cpu-type vexriscv_smp --cpu-variant linux --build --with-spi-sdcard --with-wishbone-memory
I tried to remove the LiteDRAM module by --with-wishbone-memory and let it run as of SMP cpu model --cpu-type vexriscv_smp The --cpu-variant linux will help to adjust the memory to load opensbi as needed to boot up the linux.
@gr8linux Hello. I follow your advice and run the command:
python -m litex_boards.targets.sipeed_tang_primer_20k --l2-size 512 --build --cpu-type vexriscv_smp --cpu-variant linux --build --with-spi-sdcard --with-wishbone-memory
But i encountered kernel panic when booting. Do you have any idea?
@gr8linux Hello. I follow your advice and run the command:
python -m litex_boards.targets.sipeed_tang_primer_20k --l2-size 512 --build --cpu-type vexriscv_smp --cpu-variant linux --build --with-spi-sdcard --with-wishbone-memory
But i encountered kernel panic when booting. Do you have any idea?
I can boot linux successfully just by this command
Hi,
I'm getting started on FPGAs (apologies in advance for noob behavior).
I've bought a Sipeed Tang Primer 20k ( https://wiki.sipeed.com/hardware/en/tang/tang-primer-20k/primer-20k.html ) and I'm trying to use LiteX on it. The board is already available in litex-boards.
I have a few issues, also found some workarounds (no 'nice' solution though, still I hope it helps), I'll report about them here. I'm open to any suggestions if I could help improve support for this board.
Sorry it is quite long, I'll try to split into separate parts. Summary :
Environment
toolchain: Gowin_V1.9.8.11_Education_linux
1. The reported amount of RAM is 256MB, but it should be 128?
Building litex for my board, with the most default settings:
python3 -m litex_boards.targets.sipeed_tang_primer_20k --build --load
I get the following output from the serial:
According to the specs ( https://wiki.sipeed.com/hardware/en/tang/tang-primer-20k/primer-20k.html ), my board is supposed to have 1Gbit / 128MB of DDR3. Why does it say 256MB...?
2. Memory initialization fails
This is the second part of the serial output:
I looked at similar github issues and attempted to fix it like this: try different latency values, calibrate, test. The tighest timing I found to work is cl=8, wcl=7. Commands below :
The script generates the following commands to write on the LiteX BIOS:
I then do
It seems to work:
Next, I tried to go further and see if I could adjust the code so I don't have to type these lines every time I load litex on the board. But I failed to find where I'm supposed to specify the cl/cwl timings in litex-boards or litedram. Any suggestions?
3. SD card init fails
Now, building with:
python3 -m litex_boards.targets.sipeed_tang_primer_20k --with-sdcard --build --load
It seems the SD card doesn't work. For the record, I'm using a 32GB Sandisk Extreme Pro from ~2017.
Obviously I cannot boot from SDCard but using --with-spi-sdcard seems to work, at least more than --with-sdcard ( see 5 below)
4. Building with --with-video-terminal fails
The following command:
python3 -m litex_boards.targets.sipeed_tang_primer_20k --with-video-terminal --build
gives the following error:
I tried to disable the check, by commenting out the lines in litex/litex/soc/cores/clock/gowin_gw1n.py, around line 186:
It seems to work on my display and I can see the text output by the LiteX BIOS. I don't know how to fix that "correctly" though. I guess I need to adjust a clock frequency somewhere?... I don't know.
5. Trying to boot linux-on-litex-vexriscv: so far, stuck at "liftoff!"
Call me optimistic, the tang primer 20k is not a supported board in linux-on-litex-vexriscv. But the similar tang nano 20k is, so I tried.
Mainly 2 methods:
Method 1:
I tried to add the following in linux-on-litex-vexriscv/make.py:
Method 2:
python3 -m litex_boards.targets.sipeed_tang_primer_20k --with-sdcard --cpu-type vexriscv --cpu-variant linux --with-video-terminal --build --load
Method 1 & 2 so far give the same result (note: LiteX BIOS's behavior with arrow keys is weird with method 1). I get the following:
Then it is stuck forever at this point. I also tried without video-terminal. But video-terminal is useful to confirm I didn't just loose the serial connection. Anyway, always the same result.
Note: SDCard (not spi) gives:
... as expected because sdcard_init fails.