Cocotb support

[gchqdev1729]

My team is starting to make use of FuseSoC. We have made extensive use of Cocotb for our testbenches, so we're looking for a good way to integrate the two. We have some workarounds that use the ModelSim backend, but they aren't ideal and don't really use the power of FuseSoC.

I've been chatting to my colleague @GCHQDeveloper992 and we think that one approach could be to write an Edalize backend for Cocotb. It would generate a Cocotb makefile and then call make and allow Cocotb to handle the simulator abstraction. This would be rather like the Quartus backend and it feels like the path of least resistance.

I've seen some comments on issues in FuseSoC and Cocotb projects that suggest others have had some thoughts about this too. (In particular @imphil.)

Here are a few of my thoughts to open a discussion. I'm keen to get something working for us fairly soon one way or another. We have time to do some development to put something together.

Example config

The Cocotb backend could be configured by quite a simple a tools section, something like this:

tools:
    cocotb:
        module: test_my_top_level.py
        simulator: modelsim

As the top level entity and its language are already known by FuseSoC/Edalize I hope it shouldn't be necesary to set them here explicitly, but that's an option.

Setting PYTHONPATH

Our testbenches use Python models of the components. They build on each other following the same hierarchy of dependencies as the HDL. That means PYTHONPATH needs to contain both the modules for the core at hand and the modules relating to dependencies. I wondered about doing this by adding Python files to the filesets, so they get copied in the usual way, and having the backend collect a list of the parent directories for addition to the path.

That works for our use case because our Python code isn't arranged in packages. I.e. adding the parent directory of each source file is exactly what we want to do. However, that isn't going to be universally correct. Maybe there's a better approach to this.

Virtual Environments

I'm in two minds about Virtual Environments. Maybe it's something to think about later when we have something working.

The motivation is that we have several testbenches that depend on having particular versions of Python libraries installed, both open source and internal. Up to now, we've managed that by having our build system set up a virtual environment before running Cocotb.

It would be possible to provide an option to set up a virtual environment in the backend before running Cocotb. One disadvantage is that we have to pip install Cocotb again, with the associated recompile of the simulator libraries (library compilation on install in the latest Cocotb is great, thanks!). Anther is increased complexity. The advantage is that the core file can then manage all the dependencies and no wrappers or external config are necessary, which is attractive to us.

Here's an example config with packages to be installed by pip listed, possibly with a version. Leaving the virtual_environment part would cause Cocotb to be called in the global environment.

tools:
    cocotb:
        module: test_my_top_level.py
        simulator: modelsim
        virtual_environment: 
            - 'scapy==2.3.4'
            - my_common_package

[olofk]

Thanks for looking into this. I have wanted a cocotb backend for a long time and I think it makes a lot of sense.

As for the technical side I have to confess that I'm not a big cocotb user myself, so I find it a bit hard to pass judgement. But I will try to engage stakeholders and let you come up with a good solution

[themperek]

@gchqdev1729 Maybe you can look at this https://github.com/themperek/cocotb-test/blob/master/cocotb_test/simulator.py but there are some changes in cocotb 1.4 that will make it all easier (some changes still to be added).

[gchqdev1729]

@olofk Thanks for getting back to me and for helping find people to decide on the best way forward.

@themperek Thanks for the link. I'll have a good look through. I'm currently using the 1.4.0.dev0 release - the new method library compilation on pip install has definitely made things easier for our existing build scripts that use virtual environments. Is there a good place to see what's planned for the 1.4 release - apologies if this is obvious, I'm new to GitHub as a contributor!

I'll start trying to implement something to help my understanding of the codebase and to see what works well. I've got a couple of colleagues interested in helping, including @GCHQDeveloper992, who's got useful experience on this project.

[themperek]

Is there a good place to see what's planned for the 1.4 release - apologies if this is obvious, I'm new to GitHub as a contributor!

Not really but I think, the final location and name of compiled libraries may still change.

[gchqdev1729]

Just a quick update since I've been silent on here for a few weeks - the current situation has delayed things somewhat! I've got a basic implementation working and running the VHDL and Verliog versions of the Cocotb adder example. It's a work in progress - there are a fair number of Cocotb features that are either required or desirable (https://docs.cocotb.org/en/latest/building.html) - but it works.

I'm pushing my work to https://github.com/gchqdev1729/edalize and I've got an example fusesoc core file in https://github.com/gchqdev1729/fusesoc-cores.

It was very pleasing to find how easy it was to write a backend and have it integrate with FuseSoC, once I'd worked out what I was doing!