ULX3S: Adding spiflash, U3 pinout error

[blakesmith]

Not sure if I should open this issue here, or on some other ULX3S repo...

I'm attempting to add the onboard SPI flash to the ULX3S board to enable my SoC to boot from flash (See initial WIP commit here: https://github.com/blakesmith/litex-boards/commit/2859975144bbb5a4a872ca690dfb7907fbd8b8af#diff-0d05f744fae28b2e373b10e5bc6eda679ce414f074620c726c5044e1071ed72eR87). Relevant snippet is here:

    # SPIFlash
    ("spiflash", 0,
        Subsignal("cs_n", Pins("R2")),
        Subsignal("clk", Pins("U3")),
        Subsignal("miso", Pins("V2")),
        Subsignal("mosi", Pins("W2")),
        Subsignal("wp", Pins("Y2")),
        Subsignal("hold", Pins("w1")),
        IOStandard("LVCMOS33")
    ),
    ("spiflash4x", 0,
        Subsignal("cs_n", Pins("R2")),
        Subsignal("clk", Pins("U3")),
        Subsignal("dq", Pins("W2", "V2", "Y2", "W1")),
        IOStandard("LVCMOS33")
    ),

I followed the schematics here, but the "U3" pin is not present in the CABGA381 package, and get a constraint error:

Info: Packing IOs..                                                                     
Info: pin 'clk25$tr_io' constrained to Bel 'X0/Y47/PIOA'.                               
Info: pin 'oled_ctl_csn$tr_io' constrained to Bel 'X0/Y89/PIOA'.                                                                                                                
Info: pin 'oled_ctl_dc$tr_io' constrained to Bel 'X0/Y92/PIOA'.                                                                                                                 
Info: pin 'oled_ctl_resn$tr_io' constrained to Bel 'X0/Y92/PIOB'.                                                                                                               
Info: pin 'oled_spi_clk$tr_io' constrained to Bel 'X0/Y92/PIOD'.                        
Info: pin 'oled_spi_mosi$tr_io' constrained to Bel 'X0/Y92/PIOC'.                       
Info: pin 'rst$tr_io' constrained to Bel 'X4/Y95/PIOA'.                                 
Info: pin 'sdram_cas_n$tr_io' constrained to Bel 'X126/Y89/PIOA'.                       
Info: pin 'sdram_cke$tr_io' constrained to Bel 'X126/Y38/PIOC'.                         
Info: pin 'sdram_clock$tr_io' constrained to Bel 'X126/Y38/PIOB'.
Info: pin 'sdram_cs_n$tr_io' constrained to Bel 'X126/Y86/PIOA'.                                                                                                                
Info: pin 'sdram_ras_n$tr_io' constrained to Bel 'X126/Y86/PIOB'.                       
Info: pin 'sdram_we_n$tr_io' constrained to Bel 'X126/Y86/PIOC'.                        
Info: pin 'serial_rx$tr_io' constrained to Bel 'X0/Y89/PIOB'.                           
Info: pin 'serial_tx$tr_io' constrained to Bel 'X0/Y56/PIOC'.                           
ERROR: IO pin 'spiflash_clk$tr
_io' constrained to pin 'U3', which does not exist for package 'CABGA381'.
ERROR: Packing design failed. 

Here's a screenshot from schematic as well:

I validated that that pin does not exist in the latest Trellis DB:

blake@blake-XPS-13-9360:~/src/prjtrellis/database/ECP5/LFE5UM5G-85F$ cat iodb.json | jq .packages.CABGA381.U3
null

I can't seem to find an ECP5 caBGA381 pinout diagram (just spreadsheets / CSVs) to see if the pin just has a different name from the footprint used on the board, from the one it's actually tied to.

Any advice on how to proceed here? This seem unlikely to be an issue in the Trellis Database (though it could be!) Happy to open this on a ULX3S repo if that's more appropriate.

Thanks so much for LiteX! I'm really enjoying exploring it so far.

[enjoy-digital]

Thanks for the feedback @blakesmith.

When adding the following code to the ULX3S target:

from litex.build.generic_platform import Pins, IOStandard
platform.add_extension([("dummy", 0, Pins("U3"), IOStandard("LVCMOS33"))])
self.comb += platform.request("dummy").eq(1)

./ulx3s.py --cpu-type=None --toolchain=trellis --build fails while ./ulx3s.py --cpu-type=None --toolchain=diamond --build succeeds.

So there indeed seems to be an issue in the Trellis Database. This should probably be reported to https://github.com/YosysHQ/prjtrellis or https://github.com/YosysHQ/nextpnr.

[blakesmith]

@enjoy-digital Thank you for helping me verify that! I probably need to just get the Lattice diamond toolchain setup, but I was hesitant to go through the full vendor sign-up, give them all my information, and download gigabytes of stuff, if it wasn't going to be useful - so thank you for helping me with that piece.

I will open an issue with prjtrellis and reference this issue. Thanks again for LiteX!

[daveshah1]

The solution here is to use a USRMCLK primitive - happy to be corrected but I think Diamond only works because it trims away the unused I/O, it would also fail if you actually tried to do anything with pin U3.

EDIT: I see you are actually able to use U3, this needs further investigation

[blakesmith]

@daveshah1 I was just about to add a comment on https://github.com/YosysHQ/prjtrellis/issues/138, do you think it's a related issue, or separate? Happy to open a separate issue on prjtrellis if you think that's the right path forward.

[daveshah1]

The workaround for now is to use a USRMCLK primitive. For some reason I thought you had to use this primitive in order to use the CLK pin (because the Lattice docs imply this) but evidently that isn't the case.

[enjoy-digital]

@blakesmith: sorry I haven't seen it was related the SPIFlash Clock when I did my first answer. Adding the USRMCLK should be handled in by the SPIFlash core: https://github.com/enjoy-digital/litex/blob/master/litex/soc/cores/spi_flash.py#L73-L77. I'll have a closer look to understand why it does not seem to be the case here.

[enjoy-digital]

@blakesmith: In fact you just need to comment or remove the clk in the pads definition. When provided, LiteX will try to use it, otherwise it will use the primitive of the device to drive the clock (https://github.com/enjoy-digital/litex/blob/master/litex/soc/cores/spi_flash.py#L53) If you want an example on ECP5, you can look at the ECP5-EVN: https://github.com/litex-hub/litex-boards/blob/master/litex_boards/platforms/ecp5_evn.py#L55-L67

[blakesmith]

@enjoy-digital That did the trick! The bitstream is now compiling. I didn't realize that removing the clk in the pads definition would make it use the internal USRMCLK primitive instead - sorry I missed that one when reading through the SPIMaster core!

I won't have my ULX3S board in hand to test for awhile. Once I do, I can verify that this actually works on the board.

I don't want this addition to be backwards incompatible with people who are already compiling this SoC in the field, who might be counting on stable addresses in the memory map. I'm going to see if I can try to get this added without "breaking" the locations in the memory map. If not, I'm considering putting a with_spiflash optional flag that defaults to false. I'll play with it some more, and once I have a board in hand to test, I can put this up for a PR.

Thanks again for the help @enjoy-digital and @daveshah1! The future of open source FPGA toolchains is incredibly bright, thanks to your hard work, and I really appreciate it!

[blakesmith]

Closing in favor of https://github.com/litex-hub/litex-boards/pull/155, everything is working now on my board. Thanks for the help @enjoy-digital and @daveshah1!