A package manager and module system for Filament

[rachitnigam]

A key part of a language ecosystem is a well thought out package manager and module system so let's build one for Filament! There is a couple of key challenges with designing something like this for Filament:

1. Supporting hardware generators. For example, it should be possible for someone to write a hardware generator in Chisel and provide a command line invocation to generate the corresponding Verilog file for a given Filament extern. We should figure out exactly what the interface is here but I'm hopeful that something like "consume Filament signatures and generate verilog files" would be a good first pass.
2. Supporting Black box IP. We should support things like Xilinx's LogiCORE IP generator. Again, it is a similar problem to hardware generator but the problem is that we don't get access to Verilog files anymore. We need to be tied to a particular compilation flow (Xilinx vs Synopsys) and the Verilog is only visible during synthesis.
3. Downloading packages. It's too early for us to build a package repository. I suggest we take a note from cargo and allow people to point us to particular github repository or even commits to grab a particular file that implements desired modules. We should probably disallow arbitrary file downloads for safety reasons.

Module System

The space of module systems is vast and complicated. I suggest we start with something simple and extensible like Rust's module and namespacing scheme. We should have the ability to describe private and public components and referring to them using the :: syntax.

@sampsyo's writeup on modules for Calyx is a good starting point for other ideas. Specifically, take a look at the "related work" links which mentions verilog package managers: https://github.com/cucapra/calyx/discussions/419

[rachitnigam]

FuseSoC provides a way to call generators. This is not quite want we want since it refers to a static configuration. We should ideally be able to compile a Filament program and extract all of the desired module. Then we can generate configurations like this for FuseSoC to do its own thing and link the Verilog together.

[rachitnigam]

The DefinitelyTyped repository from the TypeScript ecosystem is a good example of a "shims" package manager. The idea is that someone else is in-charge of actually implementing the Verilog (JavaScript in their case) and we are responsible for providing types for the modules.

Unlike Filament, TypeScript makes it very easy to call arbitrary JavaScript functions. I'm not sure if we want to allow this but something to consider.

[sampsyo]

This would be fun to brainstorm about in more detail. One thing I'd be particularly interested in considering is what should be Filament-specific vs. what should be more general and happen at a lower level (e.g., Calyx or FuseSoC for RTL). For example, it seems clear that the concept of a "module" and the namespace resolution features within it need to be Filament-specific. But there may be a more general notion about this "metaprogrammed packaging and distribution" that is relevant beyond Filament and might benefit from including other stuff with a similar flavor. For instance, when Filament wants to do whole-program parameter tuning, it definitely needs to see all the Filament code for the whole project. But if it only wants to tune some things and obtain other things as "black box" results of generation from parameters, then maybe those deserve a more general distribution format.

I know it seems crazy, but I kinda think doing a slam-dunk job on this sort of thing could very quickly turn into a full-fledged research project of its own…