[fud2] Switch to custom testbench generation

[ayakayorihiro]

This is a tracking issue for an effort to switch Calyx's testbenching.

Background

Currently, all Calyx-compiled Verilog programs use the same standalone testbench. The downside of this setup is that Calyx's Verilog backend needs to generate code for setting up memories (readmemh/writememh/etc), and other potential routes of obtaining Verilog (like FIRRTL) may not be able to do this. (We also want to separate out the memory initialization to the testbench, since the design itself should be independent of memory loading/writing) As part of the FIRRTL backend effort, we've created a rig that makes a custom testbench for each Calyx program. fud2 uses this custom testbench rig whenever the FIRRTL backend is involved, but it'll be great if we can make a Calyx-wide switch to using a custom testbench!

Issues and TODOs

• [ ] Simulation using the custom testbench take more cycles than using the standalone testbench (https://github.com/calyxir/calyx/issues/1956)
• [ ] The YXI backend does not emit memory information for ref cells (https://github.com/calyxir/calyx/issues/1993)
  • [x] Edited: Use the external-to-ref pass instead of a sed in fud2.
• [ ] I've introduced lots of states/transitions to the fud2 graph, so we may want the custom testbench to replace the standalone testbench.

Once the above is done, we should have a larger switch to always using the custom testbench.

• [ ] The YXI backend should be built by default.
• [ ] We should deprecate @external.
• [ ] The Verilog backend should always be in "synthesis" mode (don't produce readmemh/writememh)

[sampsyo]

This is GREAT; thanks, @ayakayorihiro! This looks like exactly the ingredients we would need.

[nathanielnrn]

This sounds cool!

If you get the chance to document the issue with the external-to-ref pass I'd be curious to hear more about it and can perhaps implement a quick fix for the bugs there.

One other thing

The YXI backend should be built by default.

I think there was hope to eventually migrate YXI outside of the calyx repo/backend, so this might be moving against that direction. I personally don't have strong feelings about this so maybe others could chime in

[sampsyo]

Yes, I think it was @rachitnigam who has stated before that he thinks that YXI JSON generation should be a separate, non-compiler tool. I don't have a strong opinion about this, TBH! Both styles seem fine to me.

(The big win here, of course, is that the AXI interface generation should be in a separate tool, which seems like an extremely worthy goal. And that goal state is enabled by YXI, wherever it goes.)

[ayakayorihiro]

Thanks @nathanielnrn !! So I figured out what I got tripped with in my initial attempts to use the external-to-ref pass, but I think it's a rather minor edge case and the way I do things in fud2 doesn't actually intersect with this error (my bad for not trying that out earlier).

When you run -p external-to-ref -b firrtl, you run into a well-formedness error on the FIRRTL backend's validation mechanism (which borrows from the Verilog backend):

$ cargo run examples/tutorial/language-tutorial-iterate.futil -p external-to-ref -b firrtl
   Compiling calyx-backend v0.7.1 (/home/ayaka/projects/calyx/calyx-backend)
   Compiling calyx v0.7.1 (/home/ayaka/projects/calyx)
    Finished dev [unoptimized + debuginfo] target(s) in 2.26s
     Running `target/debug/calyx examples/tutorial/language-tutorial-iterate.futil -p external-to-ref -b firrtl`
Error: Malformed Structure: Groups / Holes can not be turned into Verilog

But somehow if you write the output of external-to-ref to a file and run the FIRRTL backend on that, everything works?

$ cargo run examples/tutorial/language-tutorial-iterate.futil -p external-to-ref -o iter-ref.futil
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
     Running `target/debug/calyx examples/tutorial/language-tutorial-iterate.futil -p external-to-ref -o iter-ref.futil`
$ cargo run iter-ref.futil -b firrtl
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
     Running `target/debug/calyx iter-ref.futil -b firrtl`
circuit main:
    module main:
...

I'll dig into this a bit more because I'm curious what's going on, but I think this is more of a issue with the Verilog/FIRRTL backend's validation mechanism rather than external-to-ref (and also will be circumvented in fud2 since it helps to write the output to an intermediate file anyhow :) ).

[sampsyo]

I believe what's going on here is that running calyx -p external-to-ref -b firrtl runs only that pass (and then the FIRRTL backend). The rest of the compilation passes need to run to for the IR to be legal for emission.

Try something like this:

cargo run -- -p external-to-ref -p all -b firrtl examples/tutorial/language-tutorial-iterate.futil

(Note the addition of -p all.)

[ayakayorihiro]

Ohhh gotcha! That worked, thanks a lot @sampsyo :)