YosysHQ / apicula

Project Apicula 🐝: bitstream documentation for Gowin FPGAs
MIT License
462 stars 66 forks source link

Tang Nano 9K bad checksum when flashing #262

Closed stacksmith closed 2 weeks ago

stacksmith commented 1 month ago

A simple LED blinky builds with Gowin tools and flashes with openFPGALoader without a problem.

When built with nextpnr-gowin or nextpnr-himbaechel and packed with gowin_pack, openFPGALoader complains:

CRC check : FAIL
Read: 0x00004d6b checksum: 0x67b9

I also had no problems with my Tang Nano 1K board - it works fine with either toolchain. So it looks like gowin_pack writes the wrong CRC into the bitstream for GW1N-9C?

Any suggestions about how to procede are appreciated -- I am totally stuck.

Makefile:

BOARD=tangnano9k
DEVICE=GW1NR-LV9QN88C6/I5
FAMILY=GW1N-9C
PINS=9K_LED_project.cst
all: blink.fs

blink.json: src/LED.v
    yosys -p "synth_gowin -top led -json blink.json" src/LED.v

blink_pnr.json: blink.json src/$(PINS)
#   nextpnr-gowin --json blink.json --freq 27 --write blink_pnr.json --device $(DEVICE) --family $(FAMILY) --cst src/$(PINS)
    nextpnr-himbaechel --json blink.json --freq 27 --write blink_pnr.json --device $(DEVICE)  --vopt cst=src/$(PINS) --vopt family=$(FAMILY)
blink.fs: blink_pnr.json
    gowin_pack -d $(FAMILY) -o blink.fs blink_pnr.json

burn: blink.fs
    openFPGALoader -b $(BOARD)  blink.fs -f -v

clean:
    rm *.fs *.json
stacksmith commented 1 month ago

Interestingly, the lushuy labs counter example (conveniently has a pre-built .fs and a makefile) loads with the original .fs file, but when I re-make it, creates a different bitstream with a CRC error!

https://github.com/lushaylabs/tangnano9k-series-examples

yrabbit commented 1 month ago

A quick check with working versions did not show any problems with CRC - I used the Lushay example corrected for Himbaechel (Makefile only, attached). I’m also attaching the openFPGAloader log just for information about the exchange with the board of the checksum value in the fs header. I also attach FS.

Later I’ll check for release-release (for apicula and nextpnr).

image-fs.zip

vc.log

BOARD=tangnano9k
FAMILY=GW1N-9C
DEVICE=GW1NR-LV9QN88PC6/I5

all: counter.fs

# Synthesis
counter.json: counter.v
    yosys -p "read_verilog counter.v; synth_gowin -top counter -json counter.json"

# Place and Route
counter_pnr.json: counter.json
    nextpnr-himbaechel --json counter.json --write counter_pnr.json --freq 27  --device ${DEVICE} --vopt family=${FAMILY} --vopt cst=${BOARD}.cst

# Generate Bitstream
counter.fs: counter_pnr.json
    gowin_pack -d ${FAMILY} -o counter.fs counter_pnr.json

# Program Board
load: counter.fs
    openFPGALoader -b ${BOARD} counter.fs -f

.PHONY: load
.INTERMEDIATE: counter_pnr.json counter.json

https://github.com/user-attachments/assets/90065b6f-7af7-40e7-b346-cb8351b8ab5b

yrabbit commented 1 month ago

You will save me a lot of time if you send me your .json (after nextpnr-himbaechel) and .fs :smile:

pepijndevos commented 1 month ago

And version numbers would be useful too. We switched CRC implementation recently.

stacksmith commented 1 month ago

Thank you for your quick response! I just re-made counter using the Makefile provided by @yrabbit, and the result is still a CRC error:

CRC check : FAIL
Read: 0x00007dac checksum: 0x3e6d

This happens on two different Linux boxes with a fresh github install of the toolchain.

>yosys --version
Yosys 0.43+0 (git sha1 65834440a, g++ 11.4.0-1ubuntu1~22.04 -fPIC -Os)

>nextpnr-himbaechel --version
"nextpnr-himbaechel" -- Next Generation Place and Route (Version nextpnr-0.7-52-geb099a92)

gowin_pack -- can't figure out how to get the version number, but I just got it from github a couple of days ago.

Attaching .json files. counter_pnr.json counter.lushay.json counter.json

Your help is appreciated, and feel free to ask me for more details -- I am a C coder (although thoroughly unfamiliar with this project).

stacksmith commented 1 month ago

counter.fs.txt

Had to add .txt to the filename for github...

stacksmith commented 1 month ago

A̶f̶t̶e̶r̶ w̶a̶t̶c̶h̶i̶n̶g̶ t̶h̶e̶ v̶i̶d̶e̶o̶, t̶r̶i̶e̶d̶ t̶o̶ l̶o̶a̶d̶ w̶i̶t̶h̶:̶ o̶p̶e̶n̶F̶P̶G̶A̶L̶o̶a̶d̶e̶r̶ -̶c̶ f̶t̶2̶2̶3̶2̶ c̶o̶u̶n̶t̶e̶r̶.f̶s̶ A̶n̶d̶ i̶t̶ w̶o̶r̶k̶e̶d̶!̶ T̶h̶e̶ o̶r̶i̶g̶i̶n̶a̶l̶ m̶a̶k̶e̶f̶i̶l̶e̶ h̶a̶d̶ a̶ ̶-̶b̶ t̶a̶n̶g̶n̶a̶n̶o̶9̶k̶̶ p̶a̶r̶a̶m̶e̶t̶e̶r̶, w̶h̶i̶c̶h̶ s̶e̶e̶m̶s̶ t̶o̶ c̶a̶u̶s̶e̶ t̶h̶e̶ f̶a̶i̶l̶u̶r̶e̶. No, it did not work, I was wrong... See below

stacksmith commented 1 month ago

@trabucayre over at openFPGALoader repo though that -c ft2232 and -b tangnano9k should be identical! This is getting weirder by the minute.

yrabbit commented 1 month ago

well. -c ft2232 and -b tangnano9k are identical even on your sent .FS. Checked several times. So the CRC generation is most likely okay.

https://github.com/user-attachments/assets/4d530f80-fe91-448f-bd9e-e021c2c85c33

It’s good that you have two computers because next I’ll tell you fun things about that board. Try changing the USB cable - I use a cheap isolator with external power.

cable

The next point: as you may have noticed, the SD card connector is located on this board directly under the USB connector. This makes soldering the actual body part of the USB connector a very tricky task, and in most cases (I have 4 boards) the connector is not soldered in any way - from the outside it seems that it is soldered, but under a microscope you can see that these legs move there freely and the entire structure rests on the flat pins of the USB itself tires.

solder

smooth

Before the connector comes off completely, there are intermediate stages - partial contact, high resistance on some lines and other joys.

board

In general, Tangnano9k should be handled gently and it is advisable not to breathe near it.

If you have the opportunity to check on another board (at least temporarily borrowed), then let us know.

stacksmith commented 1 month ago

I was wrong, somehow. -c ft2232 does not work.

I think I must've git-reset the project and wound up with the original .fs file, and make didn't do anything, and... yeah.

So still no go.

Thanks for pointing out the poor reliability of the board, but it is obviously not an issue here -- I can very consistently load the bitstream from the repo, but not the one I build.

yrabbit commented 1 month ago

Ok. But I’m not compiling anything - I’m using the file that you sent - counter.fs.txt, which actually has a checksum of 0x3e6d and openFPGAloader says so when the -v key is specified. However, for me, the checksum is read back correctly. So there are only two things left: the openFPGAloader version and the physical wires. The image generation process has nothing to do with it.

rabbit@fly ~/tmp% openFPGALoader -V
openFPGALoader v0.11.0
rabbit@fly ~/tmp% openFPGALoader -vb tangnano9k counter.fs
Jtag frequency : requested 6.00MHz   -> real 6.00MHz
found 1 devices
index 0:
        idcode 0x100481b
        manufacturer Gowin
        family GW1N
        model  GW1N(R)-9C
        irlength 8
File type : fs
Parse file Parse counter.fs:
checksum 0x3e6d
Done
DONE
bitstream header infos
CRCCheck: ON
Compress: OFF
ConfDataLength: 712
LoadingRate: 2500000
ProgramDoneBypass: OFF
SPIAddr: 00fff000
SecurityBit: ON
idcode: 1100481b
before program sram: displayReadReg 0003f020
        Memory Erase
        Gowin VLD
        Done Final
        Security Final
        Ready
        POR
        FLASH lock
Erase SRAM before erase sram: displayReadReg 0003f020
        Memory Erase
        Gowin VLD
        Done Final
        Security Final
        Ready
        POR
        FLASH lock
pollFlag: 3f0a0 (80)
pollFlag: 3d080 (0)
pollFlag: 390a0 (20)
after erase sram: displayReadReg 000390a0
        Memory Erase
        System Edit Mode
        Gowin VLD
        Ready
        POR
        FLASH lock
pollFlag: 39020 (0)
after erase sram: displayReadReg 00039020
        Memory Erase
        Gowin VLD
        Ready
        POR
        FLASH lock
DONE
Load SRAM before write sram: displayReadReg 00039020
        Memory Erase
        Gowin VLD
        Ready
        POR
        FLASH lock
Load SRAM: [==================================================] 100.00%
Done
after write sram: displayReadReg 0003f020
        Memory Erase
        Gowin VLD
        Done Final
        Security Final
        Ready
        POR
        FLASH lock
DONE
CRC check: Success
after program sram: displayReadReg 0003f020
        Memory Erase
        Gowin VLD
        Done Final
        Security Final
        Ready
        POR
        FLASH lock

https://github.com/user-attachments/assets/34031a5e-08dc-4bc8-bdd6-ebb5d9257d44

trabucayre commented 1 month ago

@stacksmith could you confirm loading a bitstream works with gowin's tools? crc computation in openFPGALoader's fs parser hasn't been changed for a while. I have checked bitstream content and indeed the embedded CRC (line near the end starting with 0x0A) doesn't match the one computed. As mentioned in openFPGALoader's issue, 9k is a specific use case due to '0' padding.

stacksmith commented 1 month ago

@yrabbit: it cannot be a hardware issue. Using openFPGALoader,

Therefore, it has something to do with the data in the bitstream generated by yosys.

@yrabbit, are you able to flash the bitstream I built, the one that @trabucayre says has a bad CRC? What is your openFPGALoader version? Mine is v0.12.1

@trabicayre, I have no problems flashing anything built with gowin's toolchain. I cannot use gowin's flashing tool, however, and only use openFPGALoader. @pepijndevos had stated above that CRC code has been changed recently...

yrabbit commented 1 month ago

Of course I can, that's all I do - see (btw, the version of openFPGALoader is indicated there - in the first lines of the log and in the video - openFPGALoader v0.11.0) https://github.com/YosysHQ/apicula/issues/262#issuecomment-2241792448

stacksmith commented 1 month ago

Ok, that is something: your openFPGALoader v0.11.0 loads the bitstream, and mine, v0.12.1 does not!

So, would it be possible for you to try v0.12.1? I in turn can try to get an older version of openFPGALoader.

P.S. Having trouble focusing since an unfortunate IQ-loss event due to pre-vax Delta COVID.

yrabbit commented 1 month ago

Nope. Still works. Although this is not pure 0.12.1 but a master branch on commit a39524636bcc0846a799ed1fb874c891fc350c1b (https://github.com/trabucayre/openFPGALoader/commit/a39524636bcc0846a799ed1fb874c891fc350c1b)

I'll try to make a pure version with the tag 0.12.1

rabbit@fly ~/tmp% openFPGALoader -V
openFPGALoader v0.12.1
rabbit@fly ~/tmp% openFPGALoader -vb tangnano9k counter.fs
empty
Jtag frequency : requested 6.00MHz   -> real 6.00MHz
found 1 devices
index 0:
        idcode 0x100481b
        manufacturer Gowin
        family GW1N
        model  GW1N(R)-9C
        irlength 8
File type : fs
Parse file Parse counter.fs:
checksum 0x3e6d
Done
DONE
bitstream header infos
CRCCheck: ON
Compress: OFF
ConfDataLength: 712
LoadingRate: 2500000
ProgramDoneBypass: OFF
SPIAddr: 00fff000
SecurityBit: ON
idcode: 1100481b
before program sram: displayReadReg 0003f020
        Memory Erase
        Gowin VLD
        Done Final
        Security Final
        Ready
        POR
        FLASH lock
Erase SRAM before erase sram: displayReadReg 0003f020
        Memory Erase
        Gowin VLD
        Done Final
        Security Final
        Ready
        POR
        FLASH lock
pollFlag: 3f0a0 (80)
pollFlag: 3d080 (0)
pollFlag: 390a0 (20)
after erase sram: displayReadReg 000390a0
        Memory Erase
        System Edit Mode
        Gowin VLD
        Ready
        POR
        FLASH lock
pollFlag: 39020 (0)
Load SRAM before write sram: displayReadReg 00039020
        Memory Erase
        Gowin VLD
        Ready
        POR
        FLASH lock
Load SRAM: [==================================================] 100.00%
Done
after write sram: displayReadReg 0003f020
        Memory Erase
        Gowin VLD
        Done Final
        Security Final
        Ready
        POR
        FLASH lock
DONE
CRC check: Success
after program sram: displayReadReg 0003f020
        Memory Erase
        Gowin VLD
        Done Final
        Security Final
        Ready
        POR
        FLASH lock

https://github.com/user-attachments/assets/c5d75b61-7b60-4e42-baee-d2ed4fba4582

stacksmith commented 1 month ago

Oh, you are loading it into sram!

That works for me.

Could you please try to address the actual issue? And I appreciate the effort of making videos, but they are distracting and not neccessary.

My makefile listed in the original message has basically

openFPGALoader -b tangnano9k  blink.fs -f -v

Thanks

yrabbit commented 1 month ago

Oh! Well, that means I was looking in the wrong place :) Flash writing doesn't work, that's for sure.

Probably CRC is not checked with SRAM?

stacksmith commented 1 month ago

I don't know if it's related, but I don't get any FLASH lock messages when configuring via sram

yrabbit commented 1 month ago

I'm no help here. At first glance, the CRC check function is the same and is called for both SRAM and flash. It should work the same way, right?

https://github.com/trabucayre/openFPGALoader/blob/a39524636bcc0846a799ed1fb874c891fc350c1b/src/gowin.cpp#L365-L393

stacksmith commented 1 month ago

I haven't had a chance to look at any code, hoping that someone here would instantly know where to look based on the symptoms... But yes, ideally the same function should work the same way. Unless the universe is broken.

stacksmith commented 1 month ago

OK, this is getting really weird. When I thought sram configuration worked, I wasn't quite right -- it didn't error, but perhaps did nothing. I can't get anything consistent when using yosys:

This is truly frustrating and unworkable... Am I really the only one trying to use this board?

pepijndevos commented 1 month ago

I think most of us use SRAM most of the time for testing. So if it's indeed the case that SRAM works but flash doesn't for this particular board that is something that could have slipped past. Although it is surprising since flash definitely worked last time I tried on the Trenz board which has practically the same chip.

If someone wants to debug this the best way would be to take a line from a Gowin bitstream except for the last 16 bits (iirc) and compute the CRC and make sure it matches what Gowin generates.

trabucayre commented 1 month ago

In fact, for openFPGALoader, SRAM and flash differs by these lines. I don't remember exactly why this piece of code was added (maybe for something related to usercode) but checksum is overwritten by the one computed by openFPGALoader: it's maybe why bitstream is working with SRAM but not with FLASH. @pepijndevos I will try to find why CRC differs between gowin, openFPGALoader and apicula.

yrabbit commented 1 month ago

This is truly frustrating and unworkable... Am I really the only one trying to use this board?

The inside of this board is even more fun than the outside - such intricate routing of high-speed clocks, for example, I have not seen in any model, either younger or older. It requires special attention and many experiments. I use SRAM because if I flashed it every 3 minutes I would just kill the flash and I would have to buy a new board. Well, that’s why it never occurred to me that you originally meant flash :)

trabucayre commented 1 month ago

@yrabbit it make sense to limit flash access yes. I must admit most of the time I limit flash write bur to validate some algo I have no choice ;-)

trabucayre commented 1 month ago

Also, using SRAM is much faster to debug/test code

stacksmith commented 1 month ago

I am not able to load SRAM consistently. I get no errors, but the board sometimes just falls back on the flash bitstream, or ignores the new bitstream. I will try to figure out the failure modes.

stacksmith commented 1 month ago

@trabucayre, SRAM is certainly great for developing, but I would hate to finish a project and not be able to flash it!

stacksmith commented 1 month ago

@yrabbit, is it a two-layer board? I am away from my lab and don't have any 'scopes, micro or oscillo... I can see you are a fan of Tang's design sensibilities, but they kind of match those of Gowin I suppose. Personally I prefer smaller/older Xilinx devices myself (I miss instantiating the SRL16 primitive dearly), but there isn't much in the $20 price range for Xilinx boards.

yrabbit commented 1 month ago

@stacksmith Well, as far as I can see, yes, at least 2 layers.

Fan? Well, I have a couple of Tang boards :) And I kind of have to figure out how they work inside.

boards

stacksmith commented 1 month ago

I was missing a 'P' in the Makefile's DEVICE equate. Now sram works fine, and flashing seems to work as well, except I still get the CRC check: FAIL message, but the bitstream flashes and loads.

PS: Thinking outloud in #264 helped... IQ=50.

yrabbit commented 1 month ago

Yes, this makes sense - these cases differ in pins and all (almost) your problems were associated with the occupation of the DONE pin (IOL14A), which is used to signal the end of the FPGA configuration (or something like that). Using it as a GPIO is a very unhealthy activity :)

done

stacksmith commented 1 month ago

It still complains about the CRC though.

yrabbit commented 1 month ago

Well, we could use your help and your programming abilities - here is a piece of code that calculates and writes a checksum in Apicula. Ignore compress - no one uses compression. https://github.com/YosysHQ/apicula/blob/fc1121f3dd5969a2c0f957fb0fe79b4f18c35d58/apycula/bslib.py#L112-L131

stacksmith commented 1 month ago

blechh... Python! As a card-carrying Smug Lisp Programmer, looking at it makes me vomit in my mouth a little. Not nearly enough parens! But seriously, I have managed to not know anything about Python except that when something fails, it's because of Python code or some breaking Python library update deep inside.

Regarding compression -- these bitstreams are some of the most verbose I've ever seen, containing megabytes of ASCII 0 and 1 characters! WTF?

yrabbit commented 1 month ago

Well then find all the bitmatrix.py files on your computer and replace them with the attached one. Has the situation with flashing the boards changed?

bm.zip

trabucayre commented 1 month ago

@yrabbit I confirm with your fix: the checksum computed in header_footer method matches the result when numpy is used, and also the crc computed by openFPGALoader

yrabbit commented 1 month ago

Great. Then the chances that @stacksmith will be able to use the flush are high, so I do the PR.

trabucayre commented 1 month ago

Yes. I have build a serie of bitstream using gowin's tools to compare checksum between this tool and openFPGALoader: each time there was the same value, so it's possible (but maybe I'm wrong) to consider openFPGALoader as a reference. If the crc is correctly computed by apicula (i.e. match) 1/ no more error at the end of load/flash sequence, 2/ the FPGA will be happy when it will computing internally the crc and comparing with the one in the bitstream.

fedor-goncharov commented 3 weeks ago

well. -c ft2232 and -b tangnano9k are identical even on your sent .FS. Checked several times. So the CRC generation is most likely okay. crc.mp4

It’s good that you have two computers because next I’ll tell you fun things about that board. Try changing the USB cable - I use a cheap isolator with external power.

cable

The next point: as you may have noticed, the SD card connector is located on this board directly under the USB connector. This makes soldering the actual body part of the USB connector a very tricky task, and in most cases (I have 4 boards) the connector is not soldered in any way - from the outside it seems that it is soldered, but under a microscope you can see that these legs move there freely and the entire structure rests on the flat pins of the USB itself tires.

solder

smooth

Before the connector comes off completely, there are intermediate stages - partial contact, high resistance on some lines and other joys.

board

In general, Tangnano9k should be handled gently and it is advisable not to breathe near it.

If you have the opportunity to check on another board (at least temporarily borrowed), then let us know.

Hi there, I have two boards - tang nano 9k and 4k, I am under Linux Ubuntu 23.10, use openFPGALoader to load bitstreams and use oss-cad-suite from Lushay Labs (includes Yossys/apicula). By now :

board 9K is a total brick - started with being unstable with CRC errors, now I cannot erase SRAM and flash memory even through Gowin EDA - Programmer

board 4K - works but randomly spits out CRC check fails - loading to SRAM always works, saving to flash memory works very randomly : I have a fixed .fs file and just call openFPGALoader (version 0.12.1) to flash it openFPGALoader -b tangnano9k -f uart_4k.fs (it is a small UART implementation using GPIOs instead of USB port because 4K has no UART dedicated pins) Literally, I run 5 times the above command, 4 times it tries to flash the board and fails with CRC error and on 5th time it succeeds.

Please, someone, tell me how this can be possible if not just shitty soldering / no soldering of the usb port. Maybe I simply don't understand what CRC is about, but I thought that it could not change from flash to flash if the bitstream is the same ....

yrabbit commented 3 weeks ago

The latest changes in Apicula related to the calculation of the control amount do not affect Tangnano4k - these chips have the number of bit in the row at a multiple of 8, unlike Tangnano9k. So for Tangnano4k, the CRC is calculated correctly and should work both for SRAM and Flash.

(But if you wish, you can try a new CRC calculation - replace all your files with this: https://github.com/yosyshq/apicula/blob/master/apycula/bitmatrix.py)

Next, you can check the connector (and partially tore it very easily), replace the USB cable with a very short one, use USB connectors directly on the motherboard, and not on the front panel (I say this for a normal large computer, not for laptops), use an USB insulator with external power, well, seek advice to openFPGAloader developers - maybe they have a couple of tips, like reducing the exchange speed.

Just out of curiosity, can I look at the sources of this UART-4 somewhere?