F4PGA/Symbiflow Backend

[Pocketkid2]

This is an ongoing project that @schafernc and I are working on as undergrad students under the supervision of @jgoeders. This is essentially our attempt at implementing #302 to rework the F4PGA/Sybmiflow Edalize backend.

F4PGA right now is a simple flow that calls Yosys for synthesis and VPR or NextPNR for place and route. It also adds some extra commands at the end for FASM and bitstream generation.

Tasklist (New, updated 12 Oct 2022):

• [x] Complete VPR and Yosys tweaks to allow for F4PGA usage
• [x] Complete utils.py tweaks to allow for F4PGA usage (allowing command variables)
• [x] Implement basic F4PGA flow with no configuration other than specifying the board info
• [x] Implement NextPNR support (Pending in #353)
• [x] Implement optional VPR arguments to override current defaults (Pending in #348)
• [ ] Implement a database for mapping a simple board name to its part/device/chip specifiers
• [ ] Implement openOCD and openFPGALoader support for the run phase
• [ ] Create tests for F4PGA flow
• [ ] Create documentation for F4PGA flow
• [ ] Implement support for other architectures

Tasklist (Old):

• [x] Finish barebones Yosys/VPR flow that can generate a working Basys3 bitstream (completed in #322)
• [x] Move all hard-coded values to parameters (completed in #322)
• [x] Add some basic documentation (completed in #322)
• [x] Add basic test (completed in #322)
• [x] Parameterize VPR modification (so that the VPR module doesn't hold F4PGA specific values) (completed in #322)
• [x] Support NextPNR (support for nextpnr-xilinx in #322)
• [x] Support downloading to board when calling the run step (openocd support for xc7 boards added in #322, switched to openfpgaloader in #327 which supports all openfpgaloader boards)
• [x] More advanced tests (Work done in #325)
• [x] More documentation with examples (Work done in #326)
• [ ] Internal database for mapping board name to arch/device/part names to simplify edalize config
• [ ] Support Quicklogic boards (needs F4PGA refactoring first?)
• [ ] Support Lattice boards
• [ ] Include optional openOCD support alongside openFPGALoader for run step
• [ ] Change yosys f4pga/symbiflow TCL script so that it replaces the need for the python script
• [ ] Upgrade internal DB to be able to download NextPNR chipDB files for a board
• [ ] Upgrade internal DB to be able to download default constraints files for a board

[umarcor]

For context:

[mithro]

@umarcor - It would nice if most of the scripts listed under arch-defs-install went away;

[umarcor]

@mithro, agree. Is it within the scope of this issue? The description mentions "shell wrappers", not "Python wrappers".

[Pocketkid2]

@mithro, agree. Is it within the scope of this issue? The description mentions "shell wrappers", not "Python wrappers".

@schafernc and I have been directed to make the Edalize backend call the tools directly without the need for additional python wrappers (because Edalize is already a python module). We are, however, using the existing wrappers as a guide to help us design the new F4PGA Edalize backend so that it works in roughly the same way.

[mithro]

@umarcor - Well, I was thinking that the work making edalize call the python scripts might be better spent making the python scripts disappear...

[jgoeders]

@mithro I agree.

@Pocketkid2 There are "wrappers" that are calling the individual tools, and then there are the set of Python scripts that @mithro highlighted that are doing extra processing along the way.

We will start by getting rid of the wrappers, but still call those Python scripts (just to get functionality deployed faster), but then we should tackle those Python scripts as well. We will need to look at them on a case-by-case basis to see if they should be migrated into the Edalize sub-tool, or whether it's better to make modifications to the tools themselves (eg modify VPR). I'm not familiar with these scripts enough to know.

@Pocketkid2 Can you add a new task item above with these scripts (with sub-bullet items for each script)?

[mithro]

@jgoeders - Basically, I'm pretty sure all the functionality should end directly in the VPR tool or Yosys plugin.

[jgoeders]

@jgoeders - Basically, I'm pretty sure all the functionality should end directly in the VPR tool or Yosys plugin.

OK, thanks. Good to know.

[mithro]

@kgugala / @acomodi - Antmicro where involved in creating a lot of these "extra" Python scripts, can you help advise?

[umarcor]

@Pocketkid2, there are five sets of "scripts":

• Distributed as part of the arch-defs packages (tarballs):
  • Lowest abstraction: Python scripts. Some of these do have Python dependencies, which are not explicit at the moment. See https://github.com/SymbiFlow/f4pga-arch-defs/tree/main/utils and https://github.com/SymbiFlow/f4pga-arch-defs/blob/main/requirements.txt.
  • Low abstraction: TCL scripts. I don't know whether those are plain TCL or whether they have some additional dependencies. See SymbiFlow/f4pga-arch-defs#2371.
  • Intermediate abstraction: Bash scripts. These were moved last month from arch-defs to f4pga.
• Distributed as Python package f4pga:
  • Higher abstraction: Bash scripts wrapped in Python entrypoints. For each deprecated symbiflow_* script, a Python function exists. The Python function is used to generate an entrypoint through setup.py, and it calls the corresponding shell/bash script. See https://github.com/chipsalliance/f4pga/blob/main/f4pga/wrappers/sh/__init__.py and https://github.com/chipsalliance/f4pga/blob/main/f4pga/setup.py#L53-L77.
• Highest abstraction: any workflow/pipeline management tool written in Python (Litex, f4pga, Edalize, Xeda...), or Makefiles.

Therefore, Python is both on top and below the TCL/Bash scripts. Replacing the Bash scripts with Python code is technically straightforward because those are isolated. I.e., you can fork f4pga and replace the wrapper functions with any Python code to test backwards compatibility of the Python implementation. However, the lowest abstraction python scripts were not isolated yet. Those should be identified and moved somewhere else (VPR, Yosys, plugins, f4pga...), to decouple them from the arch-defs packages/tarballs.

[olofk]

We have come full circle. I was just looking at @Pocketkid2's propsed changes to yosys regarding split_io and was wondering if my tcl/python cleanup patch (https://github.com/SymbiFlow/f4pga-arch-defs/pull/2371) was merged, when I saw it was mentioned here :)

Generally, Edalize doesn't benefit from utility tools being written in Python since it needs to make an external call to each tool anyway. This is to keep the abstraction and being able to push out each tool invocation to different hardware if so needed (think SLURM or CI pipelines).

My approach to the split_io script, which is one of those custom scripts used, was to call it from a tcl file which we tell yosys to execute. The end result is that we only call yosys once, and it will do the magic internally.

I'm not familiar enough with the other scripts, but hopefully we can dig into those as well and move bits and pieces to other places

[Pocketkid2]

My approach to the split_io script, which is one of those custom scripts used, was to call it from a tcl file which we tell yosys to execute. The end result is that we only call yosys once, and it will do the magic internally.

I fully support this answer, I'll add it to the list