Adding generic toolchain class and existing toolchain class refactoring

[trabucayre]

All toolchains classes have more or less the same structure / logic with most of the logic residing into build method. It may summarise by:

1. directories creation
2. finalize the design
3. verilog generation
4. constraints
5. timing
6. project creation
7. script creation
8. script execution

Items 1 to 3 behaviour is the same for all toolchains and consequently may be implemented with a more generic approach. The others items are specific to each toolchain.

The idea here is:

• create a generic toolchain class with 1-5 implemented as it. For others operations a call to a specific method is done
• each toolchain inherits from this generic class and provides implementation for all relevant operations

This approach allows to simplify maintains since generic part has to be updated/modified once and specifics topics about one toolchain have to be done into corresponding class.

I have pushed a first PoC / draft in this branch. Tests/updates have been done with a subset of all toolchains (icestorm, trellis, quartus).

[mithro]

I would personally like to see the toolchain stuff use edalize or silicon compiler as the backend to make it easier to improve ASIC toolchain integration and similar.

@enjoy-digital may disagree and you should go with whatever he says.

[trabucayre]

True, I haven't talk about this solution: Edalize may be a solution to address items 6 to 8. It's the solution I have, by the past, proposed to @sylefeb for Silice after discussions and reading this issue. But items 5 and 6 remains toolchain dependent and no covered by Edalize.

[enjoy-digital]

Thanks a lot @trabucayre! I'll try to have a closer look soon but already like the idea.

@mithro : Edalize could be interesting for projects that don't already have build backend and don't need fine control of the toolchain. I want LiteX to handle this part, since this often has to be fine tune to be able to do what you want with the toolchain. The work done by @trabucayre could simplify bridging to Edalize for users wanting to, but I still want to have/use our own build backends for complex projects.

[mithro]

@enjoy-digital - BTW I would like to push edalize to be able to handle your use cases.

[mithro]

@enjoy-digital - According to the developers of Silicon Compiler, it should be easily able to handle your requirements. I have no idea if this is true, but that could also be work at least an hour or two of exploration? My understanding is their documentation is pretty great - https://docs.siliconcompiler.com/en/latest/

[trabucayre]

With an approach like proposed here have multiple backends: custom, edalize and silicon compiler is possible (It's only required to take a few time to find correct approach to avoid increase complexity / reduce readability). I suppose with a CLI arg it's possible to select which backend must be used.

As noted by @enjoy-digital this issue (and corresponding work) is the first step.

Maybe I can modify one toolchain to add edalize or siliconcompiler support to see requirement to allows to have one class with multiple backend.

[enjoy-digital]

@mithro: My comment from 2020 is still valid:

Sorry but i don't think supporting Edalize natively in LiteX is a good idea: This could seem nice for simple designs where we don't need that much interaction with the toolchain (for example a LiteX target) but as soon as we'll add more complex timings constraints, import IPs, etc... it will break. Not because of Edalize, but just because we'll need to add specific code to pass the extra information to Edalize.

We also already have good support in LiteX for the toolchain we are really interested in, so native Edalize support would not add that much additional value for us. Edalize could however be interesting to experiment with the toolchains that are not already supported but it would probably be better to just add an export tool that would generate all the gateware build information to a json file (similar to whas is already done for sofware) that an eventual LiteX2Edalize tool could reuse. This would be less invasive and would also allow this information to be reused for other purposes.

From LiteX presentation you can see that the build backend is not only used to drive the toolchain but also in the early stages of design elaboration to specialize the code, Edalize/Silicon-Compiler would not handle this and handling the toolchain is the "easy" part:

I'm also not sure you realize the complexity of building/constraining correctly a complex design: I already spend weeks fine tuning Vivado scripts/constraints to be able to generate/constraint designs correctly. For Efinix FPGAs, the fine control of design specialization and build backend if even more required because ot the approach used with Efinix's Interface Designer, where we also spent weeks with @fjullien to get something usable.

So even if Edalize/Silicon-compiler supports our use-cases, we are going to keep our build-backends for the FPGA devices we are using in LiteX open-source/closed-source projects.

Now I also understand your point of view (and also understand that it would for example simplify F4PGA integration with LiteX). I also have some cases where I'd like to be able to do a quick test of a LiteX design with a toolchain/simulator supported by Edalize/Silicon-Compiler and from my 2020 comment you can see that I'm open to be able bridge with Edalize/Silicon-Compiler but not to replace our build-backends.

The work done by @trabucayre is a good first step to simplify the code base and as discussed directly with @trabucayre we have following ideas to also simplify other parts: Yosys script generation, VHDL automatic conversion through GHDL when the toolchain is not supporting VHDL, etc... with all these simplification steps done, I'm sure bridging to Edalize/Silicon-Compiler (for simple projects) will be possible without too much difficulties.

[trabucayre]

As a demonstration I have added with this commit --backend argument to select between LiteX and edalize for backend. In a second commit icestorm toolchain was modified to allows selecting edalize instead of the default custom backend. This modification was tested with an icebreaker and the serv core. It's works as expected. For icepack and nextpnr-ice40 behaviour is the same: both takes everything from CLI and edam is enough to that. But for yosys there is no ways to add commands like attrmap the only way is to provides an ys template to replace the default template and I suspect for more complex toolchain like vivado a specific tcl must be provided for all operations really specifics/optimized.

edalize is a great tool allowing to add a toolchain abstraction and works really well in most of the case, but as mentionned by @enjoy-digital in specifics case it must be helped to fit requirement.

Anyway this couple of commit shows the procedure to adds alternate backend: it's seems quite easy.