yosys 0.23: write_rtlil/read_rtlil for a hierarchical parametric design fails during hierarchy check

[ted-xie]

Version

Yosys 0.23 (git sha1 7ce5011c2, clang 14.0.6-2 -fPIC -Os)

On which OS did this happen?

Linux

Reproduction Steps

When compiling verilog files separately into RTLIL files, and then attempging to combine them and run hierarchy in a later pass, yosys emits errors related to parameters supposedly not existing.

Reproduction: Run make check from the attached tarball.

yosys_bug_rtlil_params.tar.gz

Expected Behavior

Yosys should run hierarchy -check with no issues and emit the top-level RTLIL. The equivalent workflow with read_verilog (see the check_reference target in the Makefile) works, so this seems like a problem with write_rtlil/read_rtlil.

Actual Behavior

Yosys errors-out with "ERROR: Module `mbit_flop' is used with parameters but is not parametric!", even though you can see in mbit_flop.rtlil that there is a parameter specified.

[ted-xie]

Amusingly, the yosys manual has this blurb about parametric design synthesis:

7.5 Synthesizing Parametric Designs
FIXME:
Add some information on the RTLIL::Module::derive() method and how it is used to synthesize parametric modules via the hierarchy pass

The derive() function is where the aforementioned error originates from.

[nakengelhardt]

Modules with parameters have to go back to the verilog AST to be derived for concrete parameter values, but the AST is not saved to RTLIL, so this is not expected to work.

In general, almost no passes work as expected if hierarchy -top <foo> and proc haven't been run.

[ted-xie]

Is the AST not generated during read_verilog?

In general, what I am trying to do is parallelize as much of my yosys workflow as possible. My thought process was: generating the AST (which I though read_verilog did) for compilation units is easily parallelizable, since there are no real data dependences between any of the compilation units as long as unknown modules can be blackboxed and linked together later (which I thought hierarchy would take care of). I am working on a design with hundreds of files totaling millions of lines of code, so sequentially reading all the files with read_verilog takes quite a while, and I think it should be faster.

[nakengelhardt]

The AST is generated in read_verilog, but it is an in-memory data structure only and is not exported with write_rtlil. So in the other yosys process that reads the rtlil, the AST is no longer available. To resolve the parameters you need both the instantiation and the instantiated module in the same yosys process, or you need to preprocess the verilog in some other way so that you know for each module all the parameter combinations used by the instances in the design, and can manually derive all the necessary parametrized modules in advance (chparam would allow you to do that).

[ted-xie]

Thanks @nakengelhardt

it is an in-memory data structure only and is not exported with write_rtlil

That's unfortunate; I thought by its name ("_rtlil") that it would be essentially a serialized version of the in-memory AST.

chparam would allow you to do that

That works for a small number of modules, but I have hundreds of modules, a subset of which are heavily parameterized in diverse ways (think FIFOs, CDC synchronizers, etc). For the suggested workaround to scale, I would need to know a priori what parameter combinations exist for every parametric module, which I think I can only do by running a top-level hierarchy pass, which unfortunately requires reading the entire design.

[nakengelhardt]

No, RTLIL is a netlist format - it is the internal representation that the yosys passes work on, so it represents synthesizable logic in terms of cells and wires and is quite removed from any of the frontend languages (which is not just verilog, could be vhdl, amaranth, liberty, ...). This is also why it can't represent modules with parameters: it has to have a Cell object for each instance, so if there's a generate loop that could vary the number, that can't be represented, or if a wire width is parametric, there's no way to have symbolic-width wires (they're arrays of bits, it needs to know how much memory to reserve for the array).

[ted-xie]

Okay, that makes sense I guess. I saw a couple snippets from the Yosys manual which I thought indicated that params were well supported by RTLIL, which is why I embarked upon this path:

All features of the HDL that cannot be mapped directly to these RTLIL classes must be transformed to an RTLIL-compatible representation by the HDL frontend. This includes Verilog-features such as generateblocks, loops and parameters.

each module contains a callback function into the AST frontend to generate a parametrized variation of the RTLIL::Module as needed. This callback then returns the auto-generated name of the parametrized variation of the module.

Based on our discussion so far it seems that my strategy so far will not work. Do you have any suggestions on how I can achieve that I was trying to do? Specifically, I was trying to parallelize all of the read_verilog calls in my large project so that they're not just running sequentially on a single thread.

[whitequark]

I feel that write_rtlil should fail when writing designs with unparameterized modules, or perhaps write them in a way that read_rtlil can show an error or at least a warning. (I suspect that it may not always be possible to do a hard error because of how parameterization works.)