custom parser support

[boneskull]

What is the Problem Being Solved?

LavaMoat's use-case needs support for native modules (e.g., C++ code compiled to a .node file).

Endo, however, does not need to know anything about native modules. To that end, we're proposing custom parsers.

Description of the Design

To support this, APIs such as importLocation would need to accept a new option. For example, given this interface:

interface CustomParser {
  /**
   * parser implementation; nothing special here
   */
  parser: ParserImplementation;

  /**
   * A new or known `Language`
   */
  language: Language;

  /**
   * One or more extensions which will be mapped to the value of `language`
   * no leading dot!
   */
  extensions: string[];
}

The options object could accept a value parsers of type CustomParser[].

To support this, the Language type changes to becomes a union of the known Language strings and any string. Take care not to widen the type so that it is always inferred as string.

The parser-mapping code would then add these values to the pile of data structures it uses to determine which parser to use for what file.

For now, the API should only allow new parsers. It should not allow a builtin-parser to be replaced outright. If that feature is desired, it could be added in a future PR in a backwards-compatible manner.

Package-Level Custom Policy Fields

Which packages can use which parsers can be controlled via policy.

The Policy type is flexible via type arguments, but not quite flexible enough. Currently, there's no way to add custom field to the policy at the package level (e.g., a new prop on PackagePolicy).

To avoid an explosion of custom properties (and to avoid Endo needing to know anything about native modules), PackagePolicy will now have an optional field options (of default type unknown) which can be narrowed via type argument. For example:

import {type Policy} from '@endo/compartment-mapper';

type CustomOptions = {
  foo: string;
} | number;

// the `void`s are just default args for additional PolicyItem values (packages, globals, builtins)
type CustomPolicy = Policy<void, void, void, CustomOptions>;

const policy: CustomPolicy = {
  resources: {
    butts: {
      options: {
        foo: 'pants'
      }
    }
  }
};

@endo/compartment-mapper's internal policy validator doesn't need to know anything about options other than it may exist.

Security Considerations

Certainly, whoever wields a custom parser must accept full responsibility for its misuse. The way we'd be using a custom parser is a full-blown escape hatch; a native module will throw any semblance of safety out the window.

Scaling Considerations

None--if the feature is unused.

Test Plan

• Round-trip a custom parser for a trivial filetype.
• Round-trip a custom parser for a trivial filetype, with policy.
• Assert options field does not trip up the policy validator.

Compatibility Considerations

none

Upgrade Considerations

Nothing outside of a mention in NEWS.md

cc @naugtur