MLKit assumes that it can write to the directory containing the basis library

[athas]

If you are using a system-level MLKit (ie. the basis library files are owned by root), normal users cannot pass options to MLKit that require recompilation of the basis library.

[melsman]

Normal users could potentially copy the basis library and runtime systems to another location and set the SML_LIB environment variable (or mlb-path-map) accordingly (see man mlkit), but maybe such copying could be done automatically under the hood.

We could use an ~/.mlkit/ folder for storing user-specific compiled libraries. Also, currently, the REPL creates an MLB folder in the "current directory". Perhaps a better design would be to always use the ~/.mlkit/MLB folder for the REPL.

[athas]

I don't think normal users would know how to do that. I'm also skeptical about putting things in ~/.mlkit. I would recommend creating an MLB folder next to the .mlb file being compiled. This means you cannot share compilations of the same library between multiple programs, but except for the Basis library itself, this is rare. And you can always opportunistically look for MLB files elsewhere, too, in order to support precompilation of the basis library.

[someplaceguy]

I'm running into the same issue in the latest master commit (f57e93cfdf33564babc2aa263d5c12fb123ae854) and I've tried this:

Normal users could potentially copy the basis library and runtime systems to another location and set the SML_LIB environment variable (or mlb-path-map) accordingly (see man mlkit)

However, even after doing that I still got the following error, suggesting that more than the basis library would need to be copied:

$ mlkit -no_messages -o build/gen_input_mlkit gen_input.mlb
[reading source file:   nondebug.sml]
[reading source file:   util.sml]
[reading source file:   gen_input.sml]
[reading source file:   main.sml]
/nix/store/f5dajvm7zvbd9p98kwzn9ls9w2d19sma-binutils-2.40/bin/ld: cannot find /home/username/progs/openfst/lib/runtimeSystemGC.a: No such file or directory
collect2: error: ld returned 1 exit status

Even if the required library/libraries would be copied as well, I think such a workaround is very kludgy.

I should note that other compilers, such as the Rust compiler (or rather, its cargo build system, I think) have ran into this same issue so it might be worthwhile to see how they've handled it.

I believe, but I'm not 100% sure, that such files are typically stored either:

1. Relative to the corresponding source file. This is a problem because the basis library is installed system-wide and owned by root.
2. Relative to some parent directory where the project resides (e.g. relative to the .mlb file). In this case we have 2 important .mlb files: basis.mlb and the main program's .mlb. If stored next to the basis.mlb it would be a problem because of the system-wide installation, so in this case I think it would only make sense to store the MLB directory relative to the main program's .mlb file.
3. Or, in a directory that follows the XDG Base Directory Specification. In this case, it would probably mean that the following would need to be consulted and used:

There is a single base directory relative to which user-specific non-essential (cached) data should be written. This directory is defined by the environment variable $XDG_CACHE_HOME.

If $XDG_CACHE_HOME is either not set or empty, a default equal to $HOME/.cache should be used.

In other words, this would mean storing it in $XDG_CACHE_HOME/mlkit, or if $XDG_CACHE_HOME is not defined, then $HOME/.cache/mlkit.

Either way, I would like to kindly ask that a compiler option or environment variable would be added so that the user can override where the MLB directory would be stored/consulted (this would come for free with the $XDG_CACHE_HOME solution).

The reason for my request is that besides the current problem in master, the previous 4.7.5 release of MLKit has also caused me some issues due to the interaction between the generation of the files inside MLB, the tup build system (which is very strict about keeping track of all generated files) and multiple programs being compiled in parallel sharing the same .sml files.

With the 4.7.5 release, to workaround this problem I have to create a temporary directory, copy all the .sml files over to this directory and then run mlkit there, but obviously this is not exactly the best solution...