Should imports import from a specified file, or look in the entire directory?

[divinusdracodominus]

Tree shaking is fine, but it makes more sense to do it before vpst because templar typing all that unused code adds tons of time to an already super slow compilation process, of course if it is public then it is fine to include it in .vast as it will be used by other packages perhaps, but anything not marked public just adds bloat when unused, this wouldn't add complexity, neither would it cause a decrease in programmability as for now at least, the system of importing directories could remain in place. as far as requiring mod / import file_name files are seperate entities and should in my opinion be treated as such, thus it makes more sense to require importing files (a file not a directory is the fundamental data storage unit in a filesystem, hens the name "file" system, not directory system) from rust community server: "Rust uses imports not only to resolve names, but also to resolve behavior. Which traits you have in scope determines how methods are resolved, and hiding that behind implicit imports would make it both much harder to follow where a method is coming from and can easily lead to breaking changes when conflicting traits are added implicitly"

[Verdagon]

Your issues might benefit from paragraphs and lists, it's a struggle to try to read that =P

I'm open to changes here, as long as they fit with the overall stance: Compilation speed is good, but it doesn't trump all other concerns. Compilation speed is a means to an end: developer UX. If we sacrifice developer UX somewhere else to boost compilation speed, it may or may not be a net benefit.

Another preliminary note: the current import system isn't the final one. Anything is subject to change before 1.0 (as mentioned before in https://discord.com/channels/398263331808346123/734819934760075264/840788343393615904) It's roughly modeled after the java way of doing things (minus the 1-class-per-file restriction), because that's the simplest method and fastest way to unblock module support.

Had some more ideas and thoughts on this last night:

• I suspect that Rust does this for better reasons than dead code. Since the module names are 1:1 with files, I suspect they might not even parse a file if nobody uses anything from that module, even if it's public and mentioned in lib.rs. This would be a much stronger benefit than just not including dead code.
• Loading and parsing isn't the bottleneck even before Vanguard (the typing phase currently is, and once that's fixed, LLVM will be), and become even less of a bottleneck with the rise of SSDs.

There might be a solution here that gives us the ease of the current approach, with the speed boost this issue is hoping for:

• When compilation happens, we'll make an index file that remembers what symbols are defined in what files.
• We can even pre-compute and distribute that index file, just like we'd pre-compute the .vast for certain packages.
• Next compilation, if we see that none of those files have changed and none have been added in that directory, we can reuse that file. (If this is a module from fountain, we can assume they didnt change it perhaps)

Seems promising, and there are surely other ideas that will come up before 1.0.

[divinusdracodominus]

I think that there is some agreement to be had:

1. minimize compile time.
2. make developer UX approachable as possible
3. remove arbitrary complexity

advantages of implicit imports:

1. don't get compiler errors for something in the same directory
2. don't have to declare a module tree, which can be weird for new folks to wrap their heads around
3. no accidentally forgetting to import something
4. less complex to implement in he parser and thus compiler

advantages of explicit importing same directory files:

1. makes explicit the location where an interface or struct is coming from
2. limit or remove some name collisions between things in the same directory
3. the import declaration can declare a file system independant module tree which makes moving files easier
4. can have files in a directory that aren't compile, including things that may be included as string literals

potential major concerns with not having explicit file imports:

1. lack of explicit programming (the reason dynamic typing is disliked)
2. difficult to find a given type, or where it is defined
3. compile time increase due to increased parsing and typing

alternative solutions to the above issues

1. (1,2) index.md (or similar idea)
2. tree shaking / only typing what is used

[divinusdracodominus]

still have the name collision issue if two interfaces in different files are in the same directory

[divinusdracodominus]

also private indexing issue still persists with index.vmd being used for public documentation, indexing is good I think because it makes large projects people way want to contribute to easier to digest, and get started working on/learning about

[divinusdracodominus]

"I like Rust's approach better than Java's, having used both extensively. I would prefer that Rust had gone farther: our Nickle programming language separates file and path names completely from module names. This turns out to be nice in a bunch of ways, from systems that may not even have a filesystem to systems with weird filename restrictions to importing a module with a given name and interface from one of several system directories without getting the programming language involved. But Rust's thing is livable." (Bart Massey, professor of computer science at Portland State University, Portland Oregon 97201)

[divinusdracodominus]

package == directory -> fountain.toml per directory as packages should be self contained and having all files in a package in one directory is very messy, and forces arbitrary modularity

[Verdagon]

Re: "Rust uses imports not only to resolve names, but also to resolve behavior. Which traits you have in scope determines how methods are resolved, and hiding that behind implicit imports would make it both much harder to follow where a method is coming from and can easily lead to breaking changes when conflicting traits are added implicitly"

This sounds like "spooky action at a distance", which is (imo) questionable language design. So Cassy and veber-alex on the Rust server clarified what they thought elomatreb meant by that:

Whether the trait is present or not will affect whether or not a program compiles. This would make more sense; if I'm trying to use a Spaceship as a IPrintable, then we would want an impl to be in scope.

The index approach would have similar behavior, which would make this point moot, unless I misunderstand something.

[Verdagon]

Re: "makes explicit the location where an interface or struct is coming from" I'm not sure I see that as a benefit. I'd think a package's internal file structure shouldn't leak into the package's API. (I think Rust would agree, that's why they have mod.rs re-exporting things).

Re: "limit or remove some name collisions between things in the same directory" This issue makes sense in theory, but in all my decades of programming in Java, I don't think this has ever really been a problem. If anything, it's a readability faux pas to have two same-named things anywhere near each other, and I think that counts at the directory level.

Re "the import declaration can declare a file system independent module tree which makes moving files easier" That's a good point. That is a drawback of the java/scala approach. It can be tolerated by people like me that are used to it, but yeah, it's kind of tedious.

Re: "Can have files in a directory that aren't compile, including things that may be included as string literals" I've never seen people do this, so I can't say it's a common use case. Wouldn't this be better served by renaming the file to end in something like .vale.txt ? (And if they really want syntax highlighting, every IDE has a dropdown to specify how to interpret a file)

Re: "lack of explicit programming (the reason dynamic typing is disliked)" I don't follow this line of reasoning. Magic is bad, not implicitness. Implicit is good when it's predictable. Explicit can be good as a redundancy, if the redundancy prevents errors, but I can't see what error this explicitness would prevent.

Re: "Difficult to find a given type, or where it is defined" This makes sense in theory, but in practice, I've never had a problem with this. It's not hard: it's in the same directory. We know it's in the same directory, because it's not imported from another one. If your IDE doesn't have ctrl+click, or ctrl+shift+f, then just look in the same directory.

Re: "compile time increase due to increased parsing and typing" This is a really good point. This could make or break the alternatives; if the alternatives slow down parsing too much, then we might indeed want explicit imports from the same directory.

[Verdagon]

Re: "I like Rust's approach better than Java's, having used both extensively. I would prefer that Rust had gone farther: our Nickle programming language separates file and path names completely from module names. This turns out to be nice in a bunch of ways, from systems that may not even have a filesystem to systems with weird filename restrictions to importing a module with a given name and interface from one of several system directories without getting the programming language involved. But Rust's thing is livable." (Bart Massey, professor of computer science at Portland State University, Portland Oregon 97201)"

The relevant points here are:

• Some systems don't have paths
• Some systems have weird naming restrictions

I remember Palm OS didn't really have paths in its file system, which was weird. That was like twenty years ago though. Are there many systems today that have these restrictions?

Also, Windows, Mac, and Linux IIRC all support file systems with paths. I'm not sure we want to bend over backwards for things that aren't those three. We want Vale code to run everywhere, but we don't necessarily need to be able to compile Vale code from everywhere.

[Verdagon]

Something I now understand about the "explicit" alternative: Requiring that we explicitly import things from other files in the same directory is itself sufficient to completely decouple us from the file system. (Assuming we have a package statement at the top of every file like java/scala do)

[Verdagon]

I believe the "explicit" alternative was for something like: import bork.Spaceship which would imply bork is a file, and we're getting Spaceship from it.

An interesting in-between solution ("import-self-things alternative"): import self.Spaceship this would look for a Spaceship in the same package. This, plus package statements, would completely decouple us from the file system.

Interesting, that we don't necessarily need a file to be its own namespace, to decouple us from the file system.

[Verdagon]

Wait, it just occurred to me, even if we don't require imports at all (IOW, the "implicit" alternative), we're still decoupled from the file system if we have package statements; we just look in all files that have the same package statement. So it's really the package statement that decouples us from the file system, and has nothing to do with imports.