search

f4pga / ideas

Random ideas and interesting ideas for things we hope to eventually do.
86 stars 9 forks source link

Symbiflow package manager #15

Open dvc94ch opened 6 years ago

dvc94ch commented 6 years ago

Symbiflow package manager / build system

Brief explanation

To be productive writing fpga code there needs to be a great package manager. Not sure if it is in the scope of symbiflow, but using the rust project as priori art, and the fact that most toolchains without a defacto package manager have pretty terrible and fragmented solutions, it makes sense to have a package manager be tightly coupled to a toolchain.

It should be hdl agnostic - so adding preprocessing steps before verilog should be easy. This would be the basis for integrating symbiflow into existing IDE's. To quickly get a non-sucking package manager up and running an option would be to start out with a non-sucking package manager like cargo (using it as a dependency), to get the configuration file parsing, dependency install|upgrade|lock and package publishing with little work. Then adding custom commands / replacing existing commands.

Detailed Explanation

Command line interface

symbiflow (new|install|lock|build|test|sim|synth|flash|publish|netlist|floorplan)

symbiflow flash --board icestick

Example configuration file Symbiflow.toml

[package]
name = "some name"
version = "0.1.0"
authors = ["david@craven.ch"]
repository = "https://github.com/dvc94ch/example"
license = "ISC"

[dependencies]
ip = "0.1.0"
other-ip = { git = "https://github.com/owner/repo" rev="sha1" }

[build]
language = "myhdl"

[board.icestick]
target = "ice40-1k-tf144"

[target.ice40-1k-tf144]
pnr = "arachne-pnr"

Example project layout

.
├── boards
│   └── icestick
│       ├── top.pcf
│       └── top.py
├── rtl
│   └── top.py
├── Symbiflow.toml
└── tests
mx-shift commented 6 years ago

Does something like FuseSoC meet your needs in this space?

dvc94ch commented 6 years ago

What I was looking for was https://github.com/cfelton/rhea, that's exactly what I want. A shame I already wasted three days on writing my own package manager...

if __name__ == '__main__':
    from rhea.build.boards import get_board

    brd = get_board('icestick')
    flow = brd.get_flow(top=icestick)
    flow.run()
GitHub
cfelton/rhea
rhea - A collection of MyHDL cores and tools for complex digital circuit design
mithro commented 6 years ago

While I don't want to discourage you from working on something you are finding interesting and fun, the general SymbiFlow philosophy is (where possible) to take the current best existing solution and work on improving that rather than writing stuff from scratch.

I'd encourage you to reach out to the FuseSoC developer @olofk and see if you can collaborate on that. I believe he already has a library which does exactly what you are after which is reusable outside of FuseSoC. Collaborating with @cfelton would be another awesome idea!

dvc94ch commented 6 years ago

@mithro yes that makes sense. I'm not aware of all the stuff that is available in the fpga space. I was aware of fusesoc.

I did also write a useful tool - maybe already exists - to extract a signals from a vcd dump for protocol analysis with sigrok-cli. (fails to parse if there are any vectors in the dump)

olofk commented 6 years ago

Thanks for pinging me @mithro .

Happy to see other people who see the need for proper package management. The tooling for HDL is as often light years behind the software counterparts. I agree with many other things said here, but from thinking about this problem for the past seven years I have some differing views. I agree that the PM needs to be tightly coupled to the domain. I'm aware of many other attempts reusing e.g. CMake, Bitbake, or Gradle, but the issue I see is that they all make assumptions that don't make sense for this domain and many necessary features need to be shoe horned in. On the other hand, I think it's too narrow to target a single toolchain as you might want to do synthesis + P&R with SymbiFlow, Vivado, Quartus etc, simulations with Modelsim, Verilator, Icarus, linting with SpyGlass or verilator and FV with yosys-smtbmc. We don't want separate toolchains for all of them.

Now FuseSoC is a package manager and a build system. When I started it out much of the inspiration came from Gentoo's portage, waf, scons and autotools. Nowadays I find myself looking at Cargo and Meson for inspiration. Reuse is good, so I'm trying to avoid reinventing things where I can avoid it, but defining a package format is one of those things that couldn't be avoided. One of the things I've been working on lately is the separation of the package manager and the build system (i.e. wrappers for EDA tools) with a well-defined data format to communicate between them. This means that even if you don't like FuseSoC's approach towards package management you can reuse the tool-facing code (this is what @mithro hinted at when he mentioned the library that is usable outside of FuseSoC). I already have other FOSSi projects interested in reusing the EDA tool wrappers and it has been my intention to reach out also to the MyHDL side to raise awareness.

I'm happy to discuss more, but I'm aware that I'm moving away from the topic of a SymbiFlow package manager and more to package managers for FPGAs in general so feel free to close this issue and move the discussion elsewhere

mithro commented 6 years ago

So from the FPGA toolchain side (rather than the HDL cores / libraries point of view) I had two thoughts;

It might actually be the combination of the two things (conda to provide an easy Python environment) and APIO on top of that Python environment.

drom commented 6 years ago

How about using existing package manager? I like npm. It is the most popular package manager in the world. It has ton of useful libraries already. Just pack all our EDA libraries, tools, components and designs on 'npm' ?

mithro commented 4 years ago

Few more links;