Import and module are implemented counterintuitive

[matthias-dunkel]

Currently, the import system seems to work like this: The import "path" needs to be relative to the directory you call the effekt compiler from. For example:

- root
    - foo.effekt
    - bar.effekt

foo.effekt:

module root/foo
def foo() = ()

bar.effekt

module root/bar
import root/foo
def bar() = foo()

Running this from root will with: PATH\root> effekt bar.effekt will lead to: bar.effekt:2:1: Cannot find source for root/foo

But will work when run from PATH

This is not intuitive and leads to long import and module statements.

Solution

I think the module statements should be relative to the folder the effekt file lives in. So there is a mapping between folders and modules. The module's child and parent relationship corresponds to the folder hierarchy. Import statement should import from the declared modules. The file should not need to know where the module is to import from it, otherwise it is counterintuitive to modules. Just as it works with the standard library.

[matthias-dunkel]

I am currently looking into this issue. I print every import in the Namers resolve function. I changed line 59-61 to:

 val imports = decl.imports collect {
      case im @ source.Import(path) =>
        Context.at(im) {
          println(Context.module.source.name)
          processDependency(path) }
    }

For a file that imports nothing, I expected the output to be nothing. But actually the output is:

..\effekt\libraries\js\immutable\option.effekt
..\effekt\libraries\js\immutable\list.effekt
..\libraries\js\immutable\list.effekt

Somehow, this is imported, even when not used by the file.

Also, the standard library files are resolved with an absolute path, while every other import is resolved with a relative path (relative to the working directory!). That seems odd.

[matthias-dunkel]

Tree.scala:

/**
 * The type of whole compilation units
 *
 * Only a subset of definitions (FunDef and EffDef) is allowed on the toplevel
 *
 * A module declaration, the path should be an Effekt include path, not a system dependent file path
 *
 */
case class ModuleDecl(path: String, imports: List[Import], defs: List[Def]) extends Tree
case class Import(path: String) extends Tree

How is an "Effekt include path" defined, and why should it not be a system file path?

[b-studios]

Thanks for looking into it. 👍

First of all, what is a system file path? They are slightly different between operating systems (especially Windows...). Also, the language should work on the website with a virtualized unix file system.

I have been thinking about the design of name spacing, modules, dependencies, etc. Here are two observations:

1. it is almost impossible to find a good individual solution without considering the other aspects.
2. name spaces, modules, and file paths should not be conflated (they currently are).

I don't want to belittle your approach; I think you are right and we definitely need to do something. I would be very happy if you would be interested in driving this change, but we need a somewhat holistic solution.

[b-studios]

I think we should separate (at least) two different use cases:

• "scripting use"
• "project use"

I believe (but might be wrong) that in scripting use we might require/import/include resources relatively or absolutely given source paths.

In project use, having a global project config and automatically important ALL files as specified in the project config might be a better option.

This should also interact well with generalizing paths to URIs, like gist files etc. It should also integrate well with git-based dependency management.

In script mode, I can imagine to write something like:

include git://otherproject.git#a136f451

def foo() = ...

which should result in cloning the repo somewhere, checking out the specified commit, building the project, and bringing the symbols into scope (this should also work in the REPL, btw.).

In a project mode, it might make more sense to specify this as a dependency in a config file.

[b-studios]

@jiribenes pointed out that Go had bad experiences with using git to distribute packages.

[matthias-dunkel]

I agree that we should separate by use cases. I have a hard time understanding the current parsing step. When are the import files parsed? Are they parsed somewhere here for jvm: https://github.com/effekt-lang/effekt/blob/ff76871e5971fb8a9e730ddc0bf90efcaa76ff56/effekt/jvm/src/main/scala/effekt/context/IOModuleDB.scala#L27 :

I would like to go further with the project approach like this:

1. As you suggested, having a sort of configuration file, where all modules are specified.
2. Parse every file for every module specified and create a Source for every module / file
3. The Namer should resolve name conflicts.
4. Load the symbols accordingly.

I know that is oversimplified. But I hope you get the idea.

[jiribenes]

Just to avoid confusion, before proceeding with an ad-hoc design, it would be wise to consider all of the wildly differing aspects of what a module is / can be:

• unit of compilation (think LLVM modules, allow for separate compilation)
• unit of code organization (a namespace)
• unit of code reuse / polymorphism (think ML modules)
• unit of code sharing (think packages that people can share)

In turn, this raises multiple questions such as:

• Which of those are conflated and why? What are the consequences of conflating them? (Languages that do unify them often make heavy trade-offs!)
• Which one of those is a unit of versioning code? Or is that a completely separate notion?
• What is a file? Is it one of the above or is it a completely separate notion? How does it interact with the other concepts?
• What does an import statement refer to? A file? Unit of compilation? Unit of code sharing? All of them?

Note that the decisions made around modules will have far-reaching consequences rippling into the rest of the language, so caution and proper planning must be employed.

For reference, here's a related issue in this repo with a preliminary module design: https://github.com/effekt-lang/effekt/issues/30

[b-studios]

Also see #389 for future work on imports.