wesl.toml for LSPs

[stefnotch]

We need a way for language servers to figure out what file an "absolute path" resolves to. As in, if I write import my_project::foo, which folder does the language server look at?

wesl.toml Proposal

First, a language server scans for wesl.toml files (how do they do it for Cargo.toml and package.json? Do they only scan the root folder, or..?)

The file then contains a list of dependencies. For each dependency, we have its name, and how to find it. Currently, the only valid option is with a path that is relative to the wesl.toml.

e.g.

[dependencies] # Or suggest a different name
my_lib = { path = "./src/lib.wesl" } 

The importing proposal #53 then uses these entries to start fully qualified paths.

This proposal is explicitly designed for IDEs. Bundlers are free to ignore it and may instead request the user to provide all relevant info.

Example

import my_lib::foo::bar;

1. my_lib is found in the wesl.toml. Valid start
2. The fully qualified path then got resolved to ["my_lib", "bar", "foo.wesl"].

Implementation detail

Internally, a bundler should also generate an absolute filepath that is relative to the wesl.toml. Why? If the user specifies

my_lib = { path = ./src/lib.wesl } 
utils = { path = ./src/shaders/utils.wesl }

Then utils my_lib/shaders/utils and utils should have the same absolute filesystem path. That way, the linker only needs to look at the absolute path to find out whether it's the same module or not.

[ncthbrt]

This makes a lot of sense. Though we still need to figure out how this might work with bevy's cargo shaders

[k2d222]

agreed with the proposal and @ncthbrt comment. I think it's time to try some prototypes to depend on npm/cargo packages. I would also like if the wesl.toml file was optional. An implementation could fallback to looking at the cargo.toml/package.json.

[stefnotch]

@k2d222 This current wesl.toml proposal is purely for having non-relative imports within a project. Akin to import { cat } from "@/foo" or use crate::foo::cat.

As long as one only uses relative imports, this wesl.toml wouldn't be needed.

I'm somewhat sceptical about making it the libraries part optional, because from the point of view of a language server, it's not entirely obvious which files and package managers it should check. Bonus points for complex cases like monorepos with multiple package.json files. But I also get that adding a config in two separate files is annoying. Maybe a good-enough compromise could be

1. wesl.tom file is not optional
2. wesl.toml file can "extend" another file. If I extend my package.json, then every library from there can be used in wesl.

[stefnotch]

And yes, we should definitely experiment with npm/cargo packages! :)

[k2d222]

I'm somewhat sceptical about making it the libraries part optional, because from the point of view of a language server, it's not entirely obvious which files and package managers it should check. 

99% of cases are not edge-cases, let's optimize UX for that. in those cases the package.json/cargo.toml file already contains all the information and the wesl file would be redundant. I don't think it would be too difficult for the langserver either. With your compromise, it would already have to know how to parse package.json/cargo.toml files.