search

litex-hub / linux-on-litex-vexriscv

Linux on LiteX-VexRiscv
BSD 2-Clause "Simplified" License
590 stars 175 forks source link

Add support for flashing boards #189

Closed zguig52 closed 2 years ago

zguig52 commented 3 years ago

Hi guys,

I'm working on flashspi and board ECPIX5. I had first an issue with specific clock pin as described here: https://github.com/YosysHQ/prjtrellis/issues/158

I finished by just removing the associated core clk pinout and have this finally:

SPIFlash

("spiflash", 0,
    Subsignal("cs_n", Pins("AA2")),
    Subsignal("mosi", Pins("AE2")),
    Subsignal("miso", Pins("AD2")),
    Subsignal("wp", Pins("AF2")),
    Subsignal("hold", Pins("AE1")),
    IOStandard("LVCMOS33")
),
("spiflash4x", 0,
    Subsignal("cs_n", Pins("AA2")),
    Subsignal("dq", Pins("AE2", "AD2", "AF2", "AE1")),
    IOStandard("LVCMOS33")
),

Is it the right way to do?

Second, now I would like to flash the FPGA config in it and saw that the make.py --flash is to be done in project linux on vexrisc. Have you a few guidelines/conclusions on how is planned to be implemented this function so I can try to do it?

zguig52 commented 3 years ago

What I understood yet (for validation):

zguig52 commented 3 years ago

Did something working with: 1: changing the programmer type of board ecpix5 to OpenFPGALoader (litex_boards -> platforms/ecpix5.py): def create_programmer(self): return OpenFPGALoader("ecpix5") 2: adding the flash function to openfpgaloader to make its prototype compatible with all other models (litex -> build/openfpgaloader.py): def flash(self, address, bitstream_file):

address not used, for function prototype compatibility with other flash tools yet

    cmd = ["openFPGALoader", "--board", self.board, "--write-flash", "--bitstream", bitstream_file]
    self.call(cmd)
  1. implementing flash option for linux vexrisc (make.py): In class Board: def flash(self, address, filename): prog = self.platform.create_programmer() prog.flash(address, filename) [...] In main: if args.flash: board.flash(address="0x000000", filename=os.path.join(builder.gateware_dir, soc.build_name + board.bitstream_ext))
enjoy-digital commented 3 years ago

Hi @zguig52,

thanks for looking at this. Linux-on-LiteX-VexRiscv is heavily relying on LiteX-Boards (https://github.com/litex-hub/litex-boards) and we should probably add the flash support there by adding a --flash to the targets that would Flash the bitstream in SPI-Flash (silmilar to the --load command that loads the bitstream in SRAM). For ECP5, we are already using OpenOCD to load FPGA bistreams, so if using OpenOCD is possible I would have a preference for this. Once supported in LiteX-Boards, we could reuse this in Linux-on-LiteX-VexRiscv.

zguig52 commented 3 years ago

After some digging on the net, I see that there was a pull request related here: https://github.com/quartiq/bscan_spi_bitstreams/pull/7 It is also discussed here: https://github.com/litex-hub/litex-boards/pull/61

zguig52 commented 3 years ago

I did some tests with current code for flashing through OpenOCD, with the svf file coming from @ilya-epifanov (from previous pull requests). I saw that he was also in charge of most of OpenOCD script associated. I am stuck currently with this line from lattice/programmer.py: "flash bank spi0 jtagspi 0 0 0 0 ecp5.spi0.proxy 0x32":

Debug: 135 2 command.c:143 script_debug(): command - ocd_command ocd_command type ocd_flash bank spi0 jtagspi 0 0 0 0 ecp5.spi0.proxy 0x32 Debug: 136 2 command.c:143 script_debug(): command - ocd_flash ocd_flash bank spi0 jtagspi 0 0 0 0 ecp5.spi0.proxy 0x32 Error: 138 2 tcl.c:1026 handle_flash_bank_command(): 'jtagspi' driver rejected flash bank at 0x00000000; usage: (null) Debug: 139 2 command.c:143 script_debug(): command - ocd_command ocd_command type ocd_usage flash bank Debug: 140 2 command.c:143 script_debug(): command - usage ocd_usage flash bank User : 142 2 command.c:828 command_help_show_indent(): User : 143 2 command.c:865 command_help_show(): flash bankUser : 144 2 command.c:871 command_help_show(): User : 145 2 command.c:844 command_help_show_wrap(): bank_id driver_name base_address size_bytes chip_width_bytes User : 146 2 command.c:828 command_help_show_indent(): User : 147 2 command.c:828 command_help_show_indent(): User : 148 2 command.c:828 command_help_show_indent(): User : 149 2 command.c:828 command_help_show_indent(): User : 150 2 command.c:828 command_help_show_indent(): User : 151 2 command.c:828 command_help_show_indent(): User : 152 2 command.c:844 command_help_show_wrap(): bus_width_bytes target [driver_options ...] User : 153 2 command.c:828 command_help_show_indent(): User : 154 2 command.c:865 command_help_show(): flash banksUser : 155 2 command.c:874 command_help_show(): User : 156 2 command.c:828 command_help_show_indent(): User : 157 2 command.c:865 command_help_show(): flash listUser : 158 2 command.c:874 command_help_show(): User : 159 2 command.c:828 command_help_show_indent(): User : 160 2 command.c:865 command_help_show(): mflash bankUser : 161 2 command.c:871 command_help_show(): User : 162 2 command.c:844 command_help_show_wrap(): soc_type base_addr pin_id target

Did some tests also just to flash the svf file, by commenting the specific lines where defining the target and had errors on reset halt command: Debug: 6070 2238 svf.c:1548 svf_run_command(): TDO read = 0x0200100 User : 6071 2238 command.c:544 command_print(): Time used: 0m2s118ms User : 6072 2238 command.c:544 command_print(): svf file programmed successfully for 1963 commands with 0 errors User : 6073 2238 command.c:687 command_run_line(): invalid command name "reset"

trabucayre commented 3 years ago

Maybe I'm wrong somewhere or not understood everything, but with an ECP5 (and unlike xilinx or intel) a bridge loaded in RAM in not required. After sending an specific instruction the FPGA give a direct access to the SPI flash. This way is used by openFPGALoader and @gregdavill ecpprog to write the bitstream. And, for at least, openFPGALoader, it's possible to write both firmware at the beginning of the flash and a binary file at an arbitrary location by using --offset.

gregdavill commented 3 years ago

@trabucayre Is correct the ECP5 JTAG logic contains a command that enables pass-through to the FLASH pins, so you don't have to load a bridge bitstream.

It is possible to use openOCD + a crafted SVF file. But it's quite slow, as the standard SVF protocol does not support conditional loops/branching. So after each page write and sector erase you really need to wait the maximum time as highlighted in your flash datasheet.

I have an example of python script that creates a flat SVF here: https://gist.github.com/gregdavill/4f9f536757966171ef974f98348bbacb

I also wrote a simple tool: https://github.com/gregdavill/ecpprog and I believe that openFPGALoader also works to perform FLASH writes through the JTAG interface with FT232H based programmers.

This approach is much faster because the busy flag in the flash is polled, so erase/write times are closer to typical values, rather than max.

trabucayre commented 3 years ago

I also wrote a simple tool: https://github.com/gregdavill/ecpprog and I believe that openFPGALoader also works to perform FLASH writes through the JTAG interface with FT232H based programmers.

@gregdavill true: openFPGALoader is able to perform FLASH writes with any JTAG compatible device

trabucayre commented 3 years ago

But it's quite slow, as the standard SVF protocol does not support conditional loops/branching. So after each page write and sector erase you really need to wait the maximum time as highlighted in your flash datasheet.

True again. But it's not limited to the flash: each time a wait until status bit is (un)set, SVF add a delay instead of polling for the bit.

enjoy-digital commented 3 years ago

Thanks @trabucayre, @gregdavill for the info/feedback. So let's forget OpenOCD for flashing on ECP5 then, @zguig52 we could switch the ECPIX5 to openFPGALoader or ecpprog and I'll do a pass on the different LiteX-Boards later to be sure we have flashing support for most of the supported boards.

zguig52 commented 3 years ago

Hi, thanks all for the reply and support.

I did a quick survey on openFPGALoader capabilities yet WRT litex-boards.

Native supported boards:

openFPGALoader --list-boards
board name:    cable_name
acornCle215     
arty            digilent
colorlight      
crosslinknx_evn ft2232
cyc1000         ft2232
de0nano         usb-blaster
ecp5_evn        ft2232
ecpix5          ecpix5-debug
fireant         ft232
ice40_generic   ft2232
licheeTang      anlogicCable
littleBee       ft2232
machXO2EVN      ft2232
machXO3EVN      ft2232
machXO3SK       ft2232
nexysVideo      digilent_b
pipistrello     ft2232
qmtechCycloneV  
spartanEdgeAccelBoard 
tangnano        ft2232
ulx2s           ft232RL
ulx3s           ft231X
xtrx

Supported FPGA:

openFPGALoader --list-fpga
IDCode      manufacturer  family         model               
0x81111043  lattice       ECP5           LFE5UM5G-25         
0x81112043  lattice       ECP5           LFE5UM5G-45         
0x81113043  lattice       ECP5           LFE5UM5G-85         
0xe12bb043  lattice       MachXO3LF      LCMX03LF-1300C      
0x100381b   Gowin         GW1N           GW1N-4              
0x10f0043   lattice       CrosslinkNX    LIFCL-17            
0x10f1043   lattice       CrosslinkNX    LIFCL-40-ES         
0x1111043   lattice       ECP5           LFE5UM-25           
0x1112043   lattice       ECP5           LFE5UM-45           
0x1113043   lattice       ECP5           LFE5UM-85           
0x129a043   lattice       XP2            LFXP2-8E            
0x12b5043   lattice       MachXO2        LCMXO2-7000HE       
0x12b9043   lattice       MachXO2        LCMXO2-640HC        
0x12bd043   lattice       MachXO2        LCMXO2-7000HC       
0x20f30dd   altera        cyclone 10 LP  10CL025             
0x2b150dd   altera        cyclone V      5CEA2               
0x3620093   xilinx        spartan7       xc7s15ftgb196-1     
0x362c093   xilinx        artix a7 50t   xc7a50t             
0x362d093   xilinx        artix a7 35t   xc7a35              
0x362f093   xilinx        spartan7       xc7s50              
0x37c4093   xilinx        spartan7       xc7s25              
0x900281b   Gowin         GW1N           GW1N-1              
0xa014c35   anlogic       eagle s20      EG4S20BG256         
0x1100581b  Gowin         GW1N           GW1NR-9             
0x110f1043  lattice       CrosslinkNX    LIFCL-40            
0x13631093  xilinx        artix a7 100t  xc7a100             
0x13636093  xilinx        artix a7 200t  xc7a200             
0x21111043  lattice       ECP5           LFE5U-12            
0x212e3043  lattice       MachXO3D       LCMX03D-9400HC      
0x310f0043  lattice       CertusNX       LFD2NX-17           
0x310f1043  lattice       CertusNX       LFD2NX-40           
0x41111043  lattice       ECP5           LFE5U-25            
0x41112043  lattice       ECP5           LFE5U-45            
0x41113043  lattice       ECP5           LFE5U-85            
0x44008093  xilinx        spartan6       xc6slx45            
0x612b2043  lattice       MachXO3LF      LCMX03LF-1300E      
0x612b3043  lattice       MachXO3LF      LCMX03LF-2100E      
0x612b4043  lattice       MachXO3LF      LCMX03LF-4300E      
0x612b5043  lattice       MachXO3LF      LCMX03LF-6900E      
0x612b6043  lattice       MachXO3LF      LCMX03LF-9400E      
0x612bb043  lattice       MachXO3LF      LCMX03LF-2100C      
0x612bc043  lattice       MachXO3LF      LCMX03LF-4300C      
0x612bd043  lattice       MachXO3LF      LCMX03LF-6900C      
0x612be043  lattice       MachXO3LF      LCMX03LF-9400C
zguig52 commented 3 years ago

Here the quickly done compatibility matrix (to be checked / validated / ...)

FPGACompatilbility.docx

zguig52 commented 3 years ago

Best would be to have a one tool fit all, but not sure it will be possible. Maybe the best is to put a layer of abstraction in linux on vexrisc build / flash functions (as it seems to be already ?). Then only the list of requirements might be different depending on the board (openocd / openfpgaloader / ...). Related to OpenFPGALoader VS ecpprog, what are the major pro and cons that you would see related to each solution?

As I understood / saw, there are 3 main operations to be done:

What do you think to keep the following operations and add this new one with this prototype:

Other point I would like to discuss is about the build output files gateware names, I think it might be useful to add to the file name the configuration associated (or a subfolder for each conf) to the build to avoid always having to rebuild when parameters are changing (when changing the CPU count for ex, and so on).

enjoy-digital commented 3 years ago

In fact each boards already comes with a default programmer. So it's not really an issue if the same programmer is not used for all boards, but we should still try to be consistent and avoid using too much different programmers. For Xilinx FPGA, OpenOCD is working fine and we are also using it for JTAG UART, so we are going to keep it, but for Lattice, Intel and others we could eventually change. Not sure we should keep OpenOCD to load the gateware if another programmer will be used for Flashing.

For the operations, you are right. The current load_bitstream and flash methods of GenericProgrammer allow the two first operations you are listing and part of the third one (not handling a .json file). We could probably make sure LiteX-Boards have support for flashing the gateware and once done we could add a way to flash specific images.

For the build name, we could indeed add at least add the cpu count to it (ex ecpix5_cc1, ecpix5_cc2, etc... as done for the VexRiscv cluster name) and eventually extend it to others parameters in the future.

zguig52 commented 3 years ago

OK great!

Still a few questions:

  1. Flash board gateware:

    • for implementation of load_bitstream and flash methods, shall I reuse the "generic openfpgaloader.py" or a new generic ecpprog.py, depending on the one we go for, or shall I go for a specific class in lattice/programmer.py?
    • if we stick to the actual generic openfpgaloader.py, then, do we keep the one function load_bitstream on which we add the offset parameter and at board level, this function is called with different parameters depending on the use case or we have 2/3 functions with specific names for each operations (as discussed before)

  2. Flash board firmware

    • regarding to the "flash_firmware" operation, here are the things to take in account (for verification/correction/completion):

  3. The BIOS/OpenSBI is the one in charge to read at a specific address in flash to search for a startup code: this is fixed / linked to the binary?

  4. Depending on the board/components setup, the gateware is more or less heavy in size, so how would this start address be managed dynamically? Reserving a fixed size for gateware and always loading the firmware at a specific address?

Shall we first manage the "flash_gateware" operation with current functions prototype (flash with offset 0), and flash_firmware with a specific offset or shall we go directly to 2 different functions prototypes (as there is a big coupling to all boards files to be reviewed)?

trabucayre commented 3 years ago

Thanks for my "order list" :) More seriously, some quick note for openFPGALoader:

enjoy-digital commented 3 years ago

@zguig52 We should probably first make sure we can provide a flash method for the ECPIX5 (that was your original goal, with OpenFPGALoader if you want yes), then try to extend this to the different ECP5 boards (I could do it unless you want to) and others boards (could also do it) and once done, we could see how to flash multiple images at one (with a boot.json file, we could flash both gateware, firmware and eventual configuration data at once). Working first on the other aspects give us time to think and find a nice solution for the last operation :)

mithro commented 3 years ago

@enjoy-digital - Sadly, as OpenFPGALoader is licensed under the AGPL a large number of people contributing to this repository can not use the software (or even look at the software), hence we will need to support an alternative tool as well.

trabucayre commented 3 years ago

@enjoy-digital - Sadly, as OpenFPGALoader is licensed under the AGPL a large number of people contributing to this repository can not use the software (or even look at the software), hence we will need to support an alternative tool as well.

What is the best license and the way to change this? The most important thing is not to limit access, but to allows absolutely freely use it for everyone.

mithro commented 3 years ago

What is the best license and the way to change this? The most important thing is not to limit access, but to allows absolutely freely use it for everyone.

@trabucayre -- Changing the license of a project which has been accepting contributions but not using a CLA is a fairly complicated process which is not guaranteed to be successful (see the original GCC OpenRISC support issue). I would be happy to walk you through the process.

trabucayre commented 3 years ago

Yes of course!

zguig52 commented 3 years ago

@enjoy-digital I did commits to add flash to ECPIX5 board only so you can have a check if this is what you were expecting in terms of implementation. If this is what you were expecting, I can also take in charge to add this function to all ECP5 boards, no problem (and maybe others that have not yet this method and that are supported by OpenFPGALoader)

zguig52 commented 3 years ago

@enjoy-digital - Sadly, as OpenFPGALoader is licensed under the AGPL a large number of people contributing to this repository can not use the software (or even look at the software), hence we will need to support an alternative tool as well.

@mithro, what is the issue with AGPL licence use? Here we only call the binary file to load code, so like a proprietary software, the license is not "propagated", isn't it? When reading the AGPL licence, there is something, but only related to network usage of the software. I don't think we are in this use case.

@trabucayre , best licenses to support use in both free and proprietary software are MIT / BSD from what I know. It literally doesn't oblige anybody to contribute and this comes only from free will (to the contrary of GPL). Do you agree @mithro ?

enjoy-digital commented 2 years ago

Closing since no update. The general direction for this repository if to reuse as much as possible from LiteX and LiteX-boards repository. So boards with flashing directly supported in LiteX-Boards will also be directly supported here.