Allow using libraries built into SML/NJ

jez commented 2 years ago

Environment

VS Code version: n/a
Millet version: 71988b9cfea4faed3c0aa77ad5f417895a55f99d
OS version: macOS 10.15.7

Steps to reproduce

git clone https://github.com/jez/vim-better-sml
cd multi-grep/support
mv vbs-util.cm sources.cm # See issue #3
~/github/millet/target/debgug/cli

Expected behavior

no error

Actual behavior

❯ ~/github/millet/target/debug/cli
config.sml:15:18: error[4002]: undefined structure: JSONUtil
config.sml:20:6: error[4002]: undefined structure: JSONPrinter
config.sml:26:20: error[4002]: undefined structure: JSONParser
config.sml:29:22: error[4002]: undefined value: SEL
config.sml:31:29: error[4002]: undefined value: get
config.sml:31:16: error[4002]: undefined value: asString
config.sml:34:13: error[4002]: undefined value: NotString
config.sml:35:10: error[4002]: undefined value: NotObject
config.sml:36:10: error[4002]: undefined value: FieldNotFound
config.sml:35:10: error[4012]: unreachable pattern
config.sml:36:10: error[4012]: unreachable pattern
unused-defs.sml:19:11: error[4014]: non-exhaustive binding: missing [_, _, _], [_, _], and 2 others
util.sml:9:34: error[4007]: expected vector, found string
Error: 13 errors

The sources.cm file uses two libraries that ship with SML/NJ: the basis library and the SML/NJ json library.

The CM file declares those dependencies like this:

Group is
  $/basis.cm
  $/json-lib.cm

  util.sml

  config.sml
  invert-def-use.sml
  unused-defs.sml

  main.sml

azdavis commented 2 years ago

yeah, so right now millet only ships knowing about (most of) the standard basis library (plus some extra bits like Fn and Random).

to teach it about non standard other libraries like the ones sml nj ships with, we’d need to either add their signature in this repo (which is what i did for the two just mentioned), or somehow read them from an existing sml nj installation.

i predict the latter to be highly troublesome, since i’m not even sure eg sml nj ships with the source, it seemed to only ship compiled artifacts when i was looking into adding std basis support. not to mention the problem of locating an existing sml implementation on the user’s computer.

azdavis commented 2 years ago

another idea: something like .d.sml (like .d.ts for typescript) or .smli (like .rbi for sorbet) for writing “interface files” for 3rd party deps. then it would be the responsibility of millet users to arrange the correct such files for whatever deps they use. this would probably “scale” better.

jez commented 2 years ago

I also haven't found a way to read the basis library sources out of the SML/NJ installation on disk.

However, MLton ships with a command line flag -show-basis:

mlton -show-basis foo.sml

This dumps the entire contents of the basis library into foo.sml.

There might be existing SML/NJ flags to inspect which libraries are available and their locations dynamically. You could consider emailing the SML/NJ maintainers and see if they have suggestions on how to discover that information.

In any case, these libraries basically never change—you could download the SML/NJ repo, copy the sig files from there, and include them in Millet's installation. Sorbet does something similar for the Ruby standard library.

something like .d.sml (like .d.ts for typescript) or .smli (like .rbi for sorbet) for writing “interface files” for 3rd party deps

Isn't this already what signature files are? Is there a need to duplicate that? Especially if you already have a system that is capable of processing *.sml and *.sig files, it seems redundant.

azdavis commented 2 years ago

ok, i think since the std basis libraries change infrequently we can probably include them here. though i think we’d need a config option for what sml impl you’re using since mlton and smlnj (and others) probably ship with different specific libraries.

Isn't this already what signature files are?

not quite. something i currently do for the std basis library files in tree is not check that a structure expression actually ascribes to a signature. this allows me to “conjure up” a structure that ascribes to a signature. this is handy because it allows me to skip providing definitions that would typecheck, since millet doesn’t actually care about the implementation since it doesn’t run code.

if millet allowed user defined .d.sml files or similar i’d probably have it do something similar.

place that defines the mode for not checking for ascription: https://github.com/azdavis/millet/blob/71988b9cfea4faed3c0aa77ad5f417895a55f99d/crates/statics/src/info.rs#L126
an example of that usage: https://github.com/azdavis/millet/blob/71988b9cfea4faed3c0aa77ad5f417895a55f99d/crates/analysis/src/sml/bool.sml#L29

azdavis commented 1 year ago

looks like the sml nj libs are doc'd here: https://www.smlnj.org/doc/smlnj-lib/

i might try to scrape them with something similar to what i did for the sml basis libraries: https://github.com/azdavis/sml-basis

azdavis / millet