Filament Import Redesign

[UnsignedByte]

My notebook https://github.com/cucapra/filament/discussions/158#discussioncomment-7188050 has a bunch of ideas as well, but here is a slightly more formalized version of that.

Current Syntax

• [ ] #333

Imports currently work with the following syntax:

import "path/to/file.fil";

and work similarly to Verilog's `include macro: adding every component of file.fil to the namespace (recursing in a BFS fashion). This isn't ideal for a couple reasons:

• Namespace pollution - every import must add every single component, which adds a lot of unnecessary components to the namespace (many of which end up being unused, which also adds to compile time).
• No privacy - although privacy has not been implemented yet, it is conceivable that it will be implemented in the future (as many modules might have helper components that don't need to be exposed), in which case we would want to be able to select certain components to add and not others.
• Undefined behavior - defining components that match in name to some imported component will redefine that component globally. This can be used to have some cool "abstract component"-esque behavior, but should definitely be made explicit, and at the very least have a useful error message associated with it.
• File Extensions??? - Importing path/to/file.txt and path/to/file.sv are all possible - which is generally unnecessary and adds bulk to the syntax.

New Syntax

import path::to::file::Component;
import path::to::file;

comp main<'G: 1>(
  // ...
) -> (
  // ...
) {
  c := new Component<'G>(...);
  c2 := new file::Component2<'G>(...);
}

Why ::?

Existential parameters already use the :: syntax, and I personally see them somewhat similar to rusts's Traits - where you can describe some attribute of a component without explicitly providing it. Therefore, it seems fitting to use :: (as rust does) for file imports and paths as well.

---

Overriding components?

This is very speculative, as this is quite an odd feature and isn't super necessary, but if we want to keep the same overriding syntax as before I have a couple ideas.

First, we should discuss behavior. When overwriting a component, we should do so only at that import instance. For example, if we do

import a::Comp;
// b uses a
import b;

/// override a::Comp

b should still be able to properly use a::Comp instead of the overwritten version.

Why? Prevents cascading errors - we shouldn't be breaking behavior of other libraries with an override.

override Attribute

• Uses #299

  #[override]
  comp file::Component<'G: 1>(
  // ...
  ) -> (
  // ...
  ) {
  // ...
  }

  Pros and Cons

• PRO: Simple to implement, would require us to add an extra overrides section to nodes in the file graph.
• PRO: Easy to understand; looks identical to a normal component and explicitely says exactly what is being implemented
• CON: Bulky - the entire signature (including constraints) would have to be rewritten in an override
• CON: Dangerous - overriding components that are used in other libraries (such as primitives) could lead to lots of undefined behavior and unknown errors.

Variant - override(name)

We could instead tag components with override(a::Comp) instead, and allow the component name below to be anything. This would allow for behavior like #[override(a::Comp),override(b::Comp) where we could overwrite multiple components at once (not sure why, but might be necessary).

Variant - overrideable and override

We can also specify an #[overrideable] attribute that specifies a component as able to be overwritten, which allows for better safety in terms of not allowing users to overwrite any components they want.

impl and abstract components

In file.fil:

/// Contains only the signature (and maybe a default body)
abstract comp Component<'G: 1>(
  // ...
) -> (
  // ...
);

In the main file

impl file::Component {
  // ...
}

Pros and Cons

• PRO: Easy to override - very clean syntax and doesn't require us to redefine much.
• PRO: Safe; doesn't allow users to override all components, only certain ones.
• CON: Much more complicated to implement, requires a notion of "abstract components" that must be overwritten.
  • POSSIBLE PRO: Might let us simplify the notion of extern components somewhat.
• CON: Difficult to understand - the signature is possibly hidden in some far away file, could make things difficult to read and debug.

---

Personally I'm leaning toward the first implementation, due to the simplicity of implementation and how clear it is.

[rachitnigam]

@sampsyo might enjoy thinking about this; especially the override stuff

[rachitnigam]

Related to #118

[UnsignedByte]

During our meeting yesterday, we discussed the whole importing thing and came up with a couple ideas.

We want to have some form of module system to prevent naming collisions and the like. We will likely have something like a::b;;Comp as described above, and at some level of the AST this can all be eliminated into some sort of nested mod DAG. However, this presents the question of whether we want things like a mod {} keyword (as rust has), and @rachitnigam mentioned that when printing out the resulting filament, we will want something like this anyway.

Currently, we have agreed that we don't want nested modules. This means that we don't want to allow a user to define a mod inside a file (and thus each file is exactly one module). This is useful because it will eliminate a lot of the issues to do name publicity (we can just have a simple pub/private system rather than needing something like pub(crate) and pub(super). However, we should design this in a forward-compatible manner so that sometime down the line we can change this to a more complex system as the need arises.

We also discussed the override issue, for which we currently decided that:

1. First, we should implement a version of the compiler that doesn't allow any such hacky things.
   • In this case, what we're doing in the aetherling programs right now would have to be replaced with something hacky like writing a script that pieces the corresponding files together before we have a higher module level programming system
2. Find a solution that will be safe and easy to use in the future. Some ideas we discussed were:
   • Something like OCaml's "mli" and "ml" header/source file system. This would mean placing component signatures in one file and implementations in another, and could potentially make something like the primitive system much more intuitive to work with (we can implement the components in any language, and we can swap implementations, and filament ensures the timing guarantees).
   • A provides keyword - we write filament files that, on their own, are non-compileable, and we require that somewhere in the dependency hierarchy we have some sort of provides statement that provides a certain component necessary to make it compile. This seems like more of a stop-gap and not long-term solution.
   • Looking into OCaml's modules and functors - We agreed this is definitely overkill, but it would be interesting to look into and could give us some good ideas on what we want to implement/not.
   • Maybe something like and abstract component that can be overridable, yet another option but one that will probably not be implemented.

@rachitnigam LMK if i'm missing anything else!