Porting to Ulx3s and Ulx4m Devices

[lawrie]

I am attempting to port this bootloader to the Ulx3s Ecp5 device and then to the new Ulx4m device, which has more need of a dfu bootloader.

I believe you had some discussion with Goran Mahovlic of Radiona on this. He is current getting things working on a prototype Ulx4m device.

@emard, the designer of the Ulx3s device, and I, have looked at porting your had2019-playground version to the Ulx3s, but are currently not making much progress.

So i wanted your advice on the best approach.

The main problems with the had2019 version are the different clock speed and the need for a different pll, and the use of the PSRAM.

I don't know how much the PSRAM is used. I saw that this version uses the SPRAM on the up5k.

On the Ulx3s we could either use SDRAM or BRAM depending on how much is needed.

The Ulx3s has a usb connector (us2) that is connected to the fpga. I have used that for both usb host and usb device with SaxonSoc.

I am not sure if I am best starting with this version of the had2019 version.

Am I right that the had2019 device has an 8Mhz clock, so I will need to change the pll for the 25MHz clock of the Ulx3s. The pll is hand-coded in sysmgr.v and I am not sure what parameters it needs. There are 24MHz, 48Mhz and 96MHz output clocks. I am not sure if any of them have phase shifts.

[smunaut]

So, the code executing on the RISC-V is fairly small. It needs less than 12 kbytes, I use the SPRAM on the up5k because they're convenient, but on an ECP5 the BRAM are just fine.

So the SPI PSRAM in the had badge version is just used to "save" whether this is a forced reboot to the bootloader or not. The "normal" flow is :

• FPGA load bootloader bitstream
• The bootloader checks if a button or something is pressed or not
• If it is, it enumerates on USB and starts running as a bootloader
• If it is not, it immediately asserts some IO to trigger PROGRAMN to jump to the 'BOOTNEXT' address and load the user bitstream

Now, we also want to support the user bitstream itself rebooting into the bootloader to "force" bootloader mode ( using what's called DFU runtime ). But of course the user bitstream can't just assert programn with the right bootnext because even if that indeed will load the bootloader bitstream ... it will check if the button is pressed and since it's not, will immediately go back to user mode.

So to support that use case, we needed a single bit of persistant information that the user bitstream could set and that the bootloader could check to know if this was a forced reboot or not. In the hadbadge we used a word in SPI PSRAM. But really this could be anything ... I'd have to check the board to see what can be used.

[emard]

Great! for example we have RTC chip MCP7940N, it has 64-byte available space https://hr.mouser.com/datasheet/2/268/20005010H-737592.pdf even without battery it will persist as long as USB is plugged

On 9/30/21, smunaut @.***> wrote:

So, the code executing on the RISC-V is fairly small. It needs less than 12 kbytes, I use the SPRAM on the up5k because they're convenient, but on an ECP5 the BRAM are just fine.

So the SPI PSRAM in the had badge version is just used to "save" whether this is a forced reboot to the bootloader or not. The "normal" flow is :

• FPGA load bootloader bitstream
• The bootloader checks if a button or something is pressed or not
• If it is, it enumerates on USB and starts running as a bootloader
• If it is not, it immediately asserts some IO to trigger PROGRAMN to jump to the 'BOOTNEXT' address and load the user bitstream

Now, we also want to support the user bitstream itself rebooting into the bootloader to "force" bootloader mode ( using what's called DFU runtime ). But of course the user bitstream can't just assert programn with the right bootnext because even if that indeed will load the bootloader bitstream ... it will check if the button is pressed and since it's not, will immediately go back to user mode.

So to support that use case, we needed a single bit of persistant information that the user bitstream could set and that the bootloader could check to know if this was a forced reboot or not. In the hadbadge we used a word in SPI PSRAM. But really this could be anything ... I'd have to check the board to see what can be used.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/no2fpga/no2bootloader/issues/12#issuecomment-931283340

[smunaut]

That would work nicely.

I'll have a look this weekend to make an ECP5 'skeleton' for the no2bootloader.

[emard]

Great!

Now Goran replied that on the new module there will be no RTC chip, common memory elements with ULX3S and ULX4M prototype I think are FLASH and SDRAM.

SDRAM can persist even if its refresh is temporary disabled for 0.5s when FPGA reboots, I think if 256 bytes are written to the same value eg. 0xAA about half of them (cca 128 bytes) should still keep this value 0xAA other bytes may slightly degrade.

On 9/30/21, smunaut @.***> wrote:

That would work nicely.

I'll have a look this weekend to make an ECP5 'skeleton' for the no2bootloader.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/no2fpga/no2bootloader/issues/12#issuecomment-931292533

[lawrie]

One problem with the pll on the Ulx3s, is that with a 25 MHz input clock, you don't get close to the required clock frequencies:

lawrie@ubuntu20:~/had2019-playground/projects/bootloader$ ecppll -i 25 --clkout0 96 --clkout1 48 --clkout2 24 -f pll.v
sdiv 11
sdiv 23
Pll parameters:
Refclk divisor: 6
Feedback divisor: 23
clkout0 divisor: 6
clkout0 frequency: 95.8333 MHz
clkout1 divisor: 11
clkout1 frequency: 52.2727 MHz
clkout1 phase shift: 0 degrees
clkout2 divisor: 23
clkout2 frequency: 25 MHz
clkout2 phase shift: 0 degrees
VCO frequency: 575

[smunaut]

You really only need 48 MHz.

tnt@rei ~ $ ecppll -i 25 -o 48 --highres
Pll parameters:
Refclk divisor: 5
Feedback divisor: 2
clkout0 divisor: 48
clkout0 frequency: 48 MHz
clkout1 divisor: 10
clkout1 frequency: 48 MHz
clkout1 phase shift: 0 degrees
VCO frequency: 480

I think the had design was running the flash at 2x clock to be fancy but that's not really needed and doesn't help flashing speed anyway, since you spend most of your timing waiting for the flash erase rather than waiting for the transfer of data to the flash.

[lawrie]

Thanks for that.

With that --highres parameter ecppll seems to generate invalid code:

        .CLKOP(clkout0),
        .CLKOS(clkout0),

And nextpnr complains that the pin is driven twice.

I am not sure how --highres is supposed to work if I set one of CLKOS or CLKOP blank and use the other, I don't get a 48Mhz clock.

If I omit --highres and use the 46.875 Mhz clock, things start working:

Flash Manufacturer : 9d 60 16
Flash Unique ID    : 50 31 4a 33 32 39 00 06
Command> 

And the USB device appears.

[lawrie]

I have modified the pll code a bit now and the highres clock is now working.

I will commit my changes.

I have not yet done anything about PSRAM yet, other than just omitting it.

I have changed things to use just the 7 buttons and the 8 leds that the Ulx3s has. I have kept @emard change to omit the LCD code and omit the gpio pins. The ulx3s has 56 gpio pins.

Currently bootloader.svf works, but bootloader.bit says it has no valid IDCODE.

[emard]

from 25 MHz to exact 48 MHz, 2 PLLs in series can be used for example 25 -> 60 -> 48 or 25 -> 200 -> 48

On 10/1/21, Lawrie Griffiths @.***> wrote:

I have modified the pll code a bit now and the highres clock is now working.

I will commit my changes.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/no2fpga/no2bootloader/issues/12#issuecomment-932229535

[lawrie]

This is the output from dfu-util:

lawrie@ubuntu20:~/had2019-playground-emard/projects/bootloader$ sudo dfu-util --list
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

Found DFU: [1d50:614b] ver=0005, devnum=37, cfg=1, intf=0, path="2-3", alt=5, name="Bootloader", serial="50314a3332390006"
Found DFU: [1d50:614b] ver=0005, devnum=37, cfg=1, intf=0, path="2-3", alt=4, name="Cartridge main FS region", serial="50314a3332390006"
Found DFU: [1d50:614b] ver=0005, devnum=37, cfg=1, intf=0, path="2-3", alt=3, name="Cartridge RISC-V firmware (IPL)", serial="50314a3332390006"
Found DFU: [1d50:614b] ver=0005, devnum=37, cfg=1, intf=0, path="2-3", alt=2, name="Cartridge ECP5 bitstream (SoC)", serial="50314a3332390006"
Found DFU: [1d50:614b] ver=0005, devnum=37, cfg=1, intf=0, path="2-3", alt=1, name="RISC-V firmware (IPL)", serial="50314a3332390006"
Found DFU: [1d50:614b] ver=0005, devnum=37, cfg=1, intf=0, path="2-3", alt=0, name="ECP5 bitstream (SoC)", serial="50314a3332390006"

[lawrie]

I have pushed my current changes that include @emard's changes - https://github.com/lawrie/had2019-playground/commit/451c01de2d94b5d02a6b195702a5116e6038c8ac

[smunaut]

Wow, congrats !

Yeah, zones need to be defined ... usually I just allocate a zone for the bitstream and then some 'data' zone for risc-v firmware or something. I want to get more flexible with DFuSE extension in the future but ... yet to do.

You can have a look at the ocr0.2 branch which is a port to the orange crab for some of the sw changes to get a not so complex zone setup.

[lawrie]

So, how do I download firmware with the current setup?

I tried dfu-util -D <bitstream-file> and it said there were multiple devices.

I tried adding a --serial parameter, but got the same message.

I don't know much about dfu. I have used dfu-util on various devices including the Fomu and OrangeCrab, but mainly just copied make files.

[smunaut]

-a 0 would select zone 0 which is the ECP5 bitstream

[lawrie]

lawrie@ubuntu20:~/ulx3s_examples/basics/leds$ sudo dfu-util -a 0 -D bin/toplevel.bit 
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

dfu-util: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release!!!
Opening DFU capable USB device...
ID 1d50:614b
Run-time device DFU version 0101
Claiming USB DFU Interface...
Setting Alternate Setting #0 ...
Determining device status: state = dfuIDLE, status = 0
dfuIDLE, continuing
DFU mode device DFU version 0101
Device returned transfer size 4096
Copying data from PC to DFU device
Download    [=========================] 100%       280030 bytes
Download done.

I wonder where it put it.

[smunaut]

There : https://github.com/smunaut/had2019-playground/blob/master/projects/bootloader/fw/usb_dfu.c#L157

[lawrie]

Thanks. So we should set up some zones for the Ulx3s and create a multiboot that is consistent with it.

@smunaut Is it your dfu or the Fomu one (written in migen and Litex) that is used on the OrangeCrab?

[smunaut]

The FOMU one.

[lawrie]

Thanks for all your help. Everything is more or less working now.

We just need to tidy things up and make some decisions about how we want to do things for the Ulx3s and Ulx4m.

So, I will close this issue.

[lawrie]

@smunaut We hit another issue with the bootloader on the Ulx3s. Different Ulx3s boards have different flash chips and it worked with some boards and not others.

Changing from quad to single spi for writing pages (flash_page_program instead of flash_quad_page_program) made it work on all chips.

[smunaut]

Arf :/ It's a shame that flashes with a constent command set was not used for building them ...

Do you have a list of the part numbers that were used ?

[lawrie]

@emard posted this:

openFPGALoader flash chip ID
12F 
Jedec ID          : ef
memory type       : 40
memory capacity   : 18
85F
Jedec ID          : 9d
memory type       : 60
memory capacity   : 18
EDID + CFD length : 9d
EDID              : 1860
CFD               :

The 12F worked and the 85F didn't. But my 12F has the 9d version, and didn't work.

@emard may know the manufacturers.

[emard]

flash chip type is usually defined by market availability and price at production date, I want to keep 16MB capacity and some normal manufacturer like ISSI or winbond, but every time they offer something similar but different part number.

All chips are verified to work with tools "fujprog" and esp32 "ecp5.py" that tools don't probe chip type, they use fixed flash SPI command set so we know chips are compatible for basic read and write commands

On 10/2/21, Lawrie Griffiths @.***> wrote:

@emard posted this:

openFPGALoader flash chip ID
12F
Jedec ID          : ef
memory type       : 40
memory capacity   : 18
85F
Jedec ID          : 9d
memory type       : 60
memory capacity   : 18
EDID + CFD length : 9d
EDID              : 1860
CFD               :

The 12F worked and the 85F didn't. But my 12F has the 9d version, and didn't work.

@emard may know the manufacturers.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/no2fpga/no2bootloader/issues/12#issuecomment-932805909

[emard]

The programn is pulled after dfu-util -a 5 -D bootloader.bit and bitstream starts.

But the same start doesn't happen after dfu-util -a 0 -D blink.bit can we change something in fw to make it work the same as -a 5 ?

On 10/2/21, D EMARD @.***> wrote:

flash chip type is usually defined by market availability and price at production date, I want to keep 16MB capacity and some normal manufacturer like ISSI or winbond, but every time they offer something similar but different part number.

All chips are verified to work with tools "fujprog" and esp32 "ecp5.py" that tools don't probe chip type, they use fixed flash SPI command set so we know chips are compatible for basic read and write commands

On 10/2/21, Lawrie Griffiths @.***> wrote:

@emard posted this:

openFPGALoader flash chip ID
12F
Jedec ID          : ef
memory type       : 40
memory capacity   : 18
85F
Jedec ID          : 9d
memory type       : 60
memory capacity   : 18
EDID + CFD length : 9d
EDID              : 1860
CFD               :

The 12F worked and the 85F didn't. But my 12F has the 9d version, and didn't work.

@emard may know the manufacturers.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/no2fpga/no2bootloader/issues/12#issuecomment-932805909

[lawrie]

The standard spi seems fast enough, so I don't think we will try to get quad spi working for all chips on the Ulx3s.

The latest version of the Ulx3s bootloader is @emard's fork - https://github.com/emard/had2019-playground

I expect @goran_mahovlic will attempt to get this going on the Ulx4m. That board uses a CM4 module connected to a Raspberry Pi CM4 I/O board or other CM4 carrier board. It is mainly compatible with the Ulx3s. So to make that work, all that should be required is a new lpf file, and to check that the usb connector on the carrier board works.

I will close this issue again.

[emard]

HI

I'd like if bootloader could read flash (DFU_UPLOAD), I got gcc working so I tried to code something, but source state machine is somehow difficult for me to follow

It would be great if you could help with some skeleton code for reading flash and maybe supporting to dfuse starting address option, I could make details for flash chip specific commands

On 10/3/21, Lawrie Griffiths @.***> wrote:

The standard spi seems fast enough, so I don't think we will try to get quad spi working for all chips on the Ulx3s.

The latest version of the Ulx3s bootloader is @emard's fork - https://github.com/emard/had2019-playground

I expect @goran_mahovlic will attempt to get this going on the Ulx4m. That board uses a CM4 module connected to a Raspberry Pi CM4 I/O board or other CM4 carrier board. It is mainly compatible with the Ulx3s. So to make that work, all that should be required is a new lpf file, and to check that the usb connector on the carrier board works.

I will close this issue again.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/no2fpga/no2bootloader/issues/12#issuecomment-932901462

[smunaut]

The fw used in the hadbootloader repo is kind of "historical". The one used in this repo ( no2bootloader ) already supports DFU_UPLOAD and has a new version of the usb stack as well.

[emard]

HI

Thanx for this valuable hint! I added some code and it seems to dump correct data so I'm quite happy! Only I'm unsure of this line here, is it ok this to be indexed [0]?

xfer->data = g_dfu.buf.data[0];

https://github.com/emard/had2019-playground/blob/master/projects/bootloader/fw/usb_dfu.c#L353

This DFU bootloader is really good and quality new bootloader might be even better I expect! us with ecp5 boards welcome your every new code :)

On 10/9/21, smunaut @.***> wrote:

The fw used in the hadbootloader repo is kind of "historical". The one used in this repo ( no2bootloader ) already supports DFU_UPLOAD and has a new version of the usb stack as well.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/no2fpga/no2bootloader/issues/12#issuecomment-939365898

[emard]

I have added verify-erase-write logic which minimizes flash wear. it will avoid erase and/or write if not necessary (data equal).

https://github.com/emard/had2019-playground/blob/master/projects/bootloader/fw/spi.c#L80 https://github.com/emard/had2019-playground/blob/master/projects/bootloader/fw/usb_dfu.c#L179

For repeated verify error, programming must stop but I don't know how to signal back to dfu-util so currently I'm just discarding whole core with programn signal

If you have some better suggestion how to signal to dfu-util about verify error...

On 10/10/21, D EMARD @.***> wrote:

HI

Thanx for this valuable hint! I added some code and it seems to dump correct data so I'm quite happy! Only I'm unsure of this line here, is it ok this to be indexed [0]?

xfer->data = g_dfu.buf.data[0];

https://github.com/emard/had2019-playground/blob/master/projects/bootloader/fw/usb_dfu.c#L353

This DFU bootloader is really good and quality new bootloader might be even better I expect! us with ecp5 boards welcome your every new code :)

On 10/9/21, smunaut @.***> wrote:

The fw used in the hadbootloader repo is kind of "historical". The one used in this repo ( no2bootloader ) already supports DFU_UPLOAD and has a new version of the usb stack as well.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/no2fpga/no2bootloader/issues/12#issuecomment-939365898