Project system refactor in preparation for packages work

[minestarks]

This is a large refactor of project system code in preparation for the packages work.

This aims to collapse a lot of types across the JS/VSCode/WASM/compiler layers that are similar-but-not-quite the same. I did end up getting rid of a lot of types, but in some places confusion is unavoidable (e.g. ProjectConfig vs ProgramConfig). I tried to document those places as much as possible.

Highlights

• 🎉 Get rid of the deprecated overloads in ICompiler
• 🎊 Replace a bunch of separately-passed-in callbacks in the WASM layer with a trait and eliminate a lot of layers to plumb those callbacks through ProjectLoader, LanguageService, etc. This also eliminates the need for some complex macros and manually-defined types such as type AsyncFunction<'a, Arg, Return> = Box<dyn Fn(Arg) -> PinnedFuture<Return> + 'a>
• 🍾 Define a ProgramConfig type to group the compiler-related parameters that are commonly passed into the WASM functions, so we don't have to update 1,000,000 spots when we add a new compiler setting, or, say, package support.
• 🙌 Consolidate FileSystem and FileSystemAsync
• 🥳 The manifest is now consistently parsed in the native layer, rather than the parsing sometimes being done in JS.

Changes by layer:

• qsc_project

  • Consolidate FileSystem and FileSystemAsync implementations by making one call the other. Previously, we had two copies of the implementation, one sync and one async - and they're about to get a lot more complicated with packages work. So I tried an approach where the sync implementation just wraps the async one.
  • This also means that the async feature is no longer optional in this crate.
  • Define a trait, JSProjectHost, to define the manifest/filesystem functions, instead of using a struct with callbacks. This helped get rid of a lot of boxed async function types we were having to store in different places, and replaced them with more intuitive trait impls.
  • The manifest is now being parsed in the native layer instead of JS. Hopefully this will centralize our error handling a bit and make behavior more consistent across scenarios (Python, JS). It also lays the groundwork for package dependencies, where we'll have to parse dependency manifests in the native layer anyway.
  • The project loading code now returns the name and manifest_path, values which are used in various extension features. Previously these values were being generated JS-side, but it makes sense to populate them at the point we're loading the project.

• language_service

  • Call the new find_manifest_directory host function instead of get_manifest which used to do the manifest parsing JS-side.
  • Remove the project system callbacks in favor of the unified JSProjectHost trait.
  • Remove LoadManifestResult in favor of the Project type already defined in qsc_project
  • Tests: Split out the in-memory FS used in a few different tests, to a shared module test_fs.

• npm

  • Get rid of the deprecated overloads in ICompiler. QCOM has already been updated to use the new signatures.
  • Where applicable, ICompiler and IDebugService methods all take a consistent ProgramConfig type, which contains the sources and compiler settings, which also has a straightforward conversion to the type used in the WASM crate.
  • Verified backcompat with QCOM. (IDebugService is a breaking change but it's not used anywhere but VS Code).

• pip

  • Only change here is to trivially plumb through the new resolve_path filesystem method.

• vscode

  • Fix up the various spots where the ICompiler methods were updated to take ProgramConfig.
  • We don't parse the manifest in the VS Code layer anymore. Instead we just discover it (in findManifestDirectory), and the native code does all the rest.

• wasm

  • Consolidate parameters such as sources, language_features, target_profile into one ProgramConfig type. This is almost the same as the ProgramConfig in compiler.ts , except this one is more strictly typed since backcompat isn't a concern.
  • Remove tons of code that was needed make the async JS callbacks work, replacing it with a wasm-bindgen attributes where applicable, and using a trait instead of separate callbacks.

[github-actions[bot]]

Benchmark for 2582511

Click to view benchmark

| Test | Base | PR | % | |------|--------------|------------------|---| | Array append evaluation | 331.3±1.65µs | **328.0±4.04µs** | **-1.00%** | | Array literal evaluation | 192.8±2.10µs | **180.2±4.76µs** | **-6.54%** | | Array update evaluation | **410.4±1.41µs** | 412.2±2.82µs | **+0.44%** | | Core + Standard library compilation | 23.3±0.67ms | **21.6±0.97ms** | **-7.30%** | | Deutsch-Jozsa evaluation | **5.1±0.05ms** | 5.2±0.04ms | **+1.96%** | | Large file parity evaluation | **34.2±0.10ms** | 34.5±0.45ms | **+0.88%** | | Large input file compilation | 13.9±0.30ms | **13.2±0.49ms** | **-5.04%** | | Large input file compilation (interpreter) | 53.7±1.52ms | **51.3±1.89ms** | **-4.47%** | | Large nested iteration | 32.2±0.20ms | **31.8±0.38ms** | **-1.24%** | | Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample | 1600.6±103.47µs | 1573.3±99.67µs | -1.71% | | Perform Runtime Capabilities Analysis (RCA) on large file sample | **8.0±0.17ms** | 8.2±0.17ms | **+2.50%** | | Perform Runtime Capabilities Analysis (RCA) on teleport sample | 1465.6±139.14µs | 1420.6±47.10µs | -3.07% | | Perform Runtime Capabilities Analysis (RCA) on the core and std libraries | 28.8±0.54ms | **28.3±0.39ms** | **-1.74%** | | Teleport evaluation | 90.2±4.06µs | 92.0±3.77µs | +2.00% |

[github-actions[bot]]

Benchmark for cc80fcc

Click to view benchmark

| Test | Base | PR | % | |------|--------------|------------------|---| | Array append evaluation | 332.9±2.99µs | **330.8±1.50µs** | **-0.63%** | | Array literal evaluation | 189.3±1.22µs | **186.1±1.42µs** | **-1.69%** | | Array update evaluation | 413.6±3.11µs | 412.8±6.69µs | -0.19% | | Core + Standard library compilation | 21.2±1.32ms | 20.6±0.75ms | -2.83% | | Deutsch-Jozsa evaluation | 5.1±0.05ms | 5.1±0.05ms | 0.00% | | Large file parity evaluation | **34.2±0.25ms** | 34.4±0.10ms | **+0.58%** | | Large input file compilation | 12.7±0.33ms | 12.6±0.41ms | -0.79% | | Large input file compilation (interpreter) | **49.5±1.23ms** | 51.3±3.90ms | **+3.64%** | | Large nested iteration | 32.3±0.39ms | 32.1±0.33ms | -0.62% | | Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample | 1573.6±42.19µs | 1549.3±38.20µs | -1.54% | | Perform Runtime Capabilities Analysis (RCA) on large file sample | 7.9±0.28ms | 7.9±0.14ms | 0.00% | | Perform Runtime Capabilities Analysis (RCA) on teleport sample | 1457.2±120.85µs | 1414.6±67.50µs | -2.92% | | Perform Runtime Capabilities Analysis (RCA) on the core and std libraries | 28.3±1.21ms | 28.1±0.47ms | -0.71% | | Teleport evaluation | 91.0±3.58µs | 90.2±4.19µs | -0.88% |

[github-actions[bot]]

Benchmark for 1855b7c

Click to view benchmark

| Test | Base | PR | % | |------|--------------|------------------|---| | Array append evaluation | 321.0±7.43µs | 322.1±12.42µs | +0.34% | | Array literal evaluation | 166.2±5.04µs | 163.8±4.44µs | -1.44% | | Array update evaluation | 397.8±9.00µs | 397.1±9.42µs | -0.18% | | Core + Standard library compilation | 19.7±0.23ms | 19.5±0.34ms | -1.02% | | Deutsch-Jozsa evaluation | 5.0±0.12ms | 5.0±0.11ms | 0.00% | | Large file parity evaluation | 33.7±0.65ms | 34.3±0.60ms | +1.78% | | Large input file compilation | 12.1±0.51ms | 12.1±0.22ms | 0.00% | | Large input file compilation (interpreter) | 46.4±0.70ms | 45.7±1.11ms | -1.51% | | Large nested iteration | 31.2±0.78ms | 30.7±0.67ms | -1.60% | | Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample | 1570.3±67.40µs | **1535.5±34.68µs** | **-2.22%** | | Perform Runtime Capabilities Analysis (RCA) on large file sample | 7.9±0.10ms | **7.7±0.08ms** | **-2.53%** | | Perform Runtime Capabilities Analysis (RCA) on teleport sample | 1448.5±131.92µs | **1397.5±29.01µs** | **-3.52%** | | Perform Runtime Capabilities Analysis (RCA) on the core and std libraries | 28.0±0.25ms | **27.6±0.51ms** | **-1.43%** | | Teleport evaluation | 87.4±3.92µs | 90.2±4.32µs | +3.20% |

[github-actions[bot]]

Benchmark for 833fb13

Click to view benchmark

| Test | Base | PR | % | |------|--------------|------------------|---| | Array append evaluation | 330.1±4.89µs | 328.9±2.94µs | -0.36% | | Array literal evaluation | 173.2±3.12µs | 170.2±3.18µs | -1.73% | | Array update evaluation | 414.7±4.57µs | 412.6±4.56µs | -0.51% | | Core + Standard library compilation | 23.3±1.26ms | **22.3±0.85ms** | **-4.29%** | | Deutsch-Jozsa evaluation | 5.2±0.06ms | **5.1±0.05ms** | **-1.92%** | | Large file parity evaluation | 35.2±0.21ms | **34.6±0.61ms** | **-1.70%** | | Large input file compilation | 14.2±0.57ms | **13.5±0.50ms** | **-4.93%** | | Large input file compilation (interpreter) | 58.4±2.88ms | 57.8±3.10ms | -1.03% | | Large nested iteration | 32.0±0.29ms | 32.2±0.35ms | +0.62% | | Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample | 1631.5±133.19µs | 1615.6±81.42µs | -0.97% | | Perform Runtime Capabilities Analysis (RCA) on large file sample | 8.3±0.37ms | 8.4±0.36ms | +1.20% | | Perform Runtime Capabilities Analysis (RCA) on teleport sample | 1483.7±113.57µs | 1469.7±111.90µs | -0.94% | | Perform Runtime Capabilities Analysis (RCA) on the core and std libraries | 28.8±0.37ms | 28.6±0.39ms | -0.69% | | Teleport evaluation | 92.6±4.46µs | 90.4±3.29µs | -2.38% |

[github-actions[bot]]

Benchmark for 824500e

Click to view benchmark

| Test | Base | PR | % | |------|--------------|------------------|---| | Array append evaluation | 330.3±1.42µs | **327.8±1.80µs** | **-0.76%** | | Array literal evaluation | 189.5±0.68µs | **187.0±0.90µs** | **-1.32%** | | Array update evaluation | 412.3±1.40µs | **408.7±1.74µs** | **-0.87%** | | Core + Standard library compilation | 20.5±0.83ms | 21.2±0.84ms | +3.41% | | Deutsch-Jozsa evaluation | 5.1±0.05ms | 5.1±0.06ms | 0.00% | | Large file parity evaluation | **34.2±0.09ms** | 34.5±0.32ms | **+0.88%** | | Large input file compilation | 12.8±0.37ms | 12.8±0.34ms | 0.00% | | Large input file compilation (interpreter) | **49.3±1.51ms** | 51.4±2.05ms | **+4.26%** | | Large nested iteration | 32.3±1.41ms | **32.0±0.11ms** | **-0.93%** | | Perform Runtime Capabilities Analysis (RCA) on Deutsch-Jozsa sample | 1572.4±69.50µs | 1571.0±98.17µs | -0.09% | | Perform Runtime Capabilities Analysis (RCA) on large file sample | 7.8±0.11ms | 7.9±0.11ms | +1.28% | | Perform Runtime Capabilities Analysis (RCA) on teleport sample | 1429.9±42.37µs | 1410.0±36.88µs | -1.39% | | Perform Runtime Capabilities Analysis (RCA) on the core and std libraries | 28.2±0.69ms | 28.0±0.29ms | -0.71% | | Teleport evaluation | 91.8±10.28µs | 91.3±3.76µs | -0.54% |