Make it possible to use different packages

[mbrock]

Packages exist but they're not usable by Wisp code.

• [x] FOO:BAR syntax in the reader
• [x] dynamic variable *PACKAGE*
• [x] DEFPACKAGE
• [x] IN-PACKAGE (special form handled in the reader)
• [x] symbol resolution in evaluator

[mbrock]

hmm, it's a bit weird to do IN-PACKAGE as a special thing handled by the reader

rather we should always read one expression, evaluate it, and then read the next one?

so we shouldn't have a function like readMany

we should have a read function that works on buffered streams

[mbrock]

the Zig reader code already works with buffered streams, but only within single calls; those buffered streams are never returned or stored

so should we store such streams in the Wisp heap? what about serialization, etc?

well, what does it mean to serialize input streams? if the program is waiting for stdin and you save a core, how do you resume? probably that stream must be marked as invalid and recreated...

or maybe if the buffer is empty, and the stream is "well-known" like stdin, then it can simply be recreated: this would work for e.g. the REPL but might also be error-prone and confusing

a stream is kind of like an ext value except in Zig, not in JavaScript

this applies for file streams but it's actually totally different for string-reading streams which can be perfectly serialized and should just be Lisp values

[mbrock]

actually the REPL is the only place where we read forms from a file stream and there we already read a single form at a time, so there's no real problem?

there's Heap.load which does readMany and then iterates over the result to evaluate each top-level form — this should instead read a form at a time, and this would take care of loading the standard library, etc

oh and in server.js we're actually wrapping the files in (do ...) or (async ...) which is interesting because that wouldn't work with the Common Lisp style of IN-PACKAGE, you can't switch package in the middle of a form... but maybe you should be able to do that...

[mbrock]

so I'm leaning now towards actually introducing a new special syntax that changes the reader's current package, kinda block-scoped so you could have it in the middle of a list like

(list x #(in-package foo) y) => (WISP:X FOO:Y)

this seems like a good improvement, it's always nice to be able to scope things like this

we can use a similar mechanism for local symbol imports, aliases, etc

[mbrock]

hmm this is a bit obscure but maybe important

imagine loading a library in the middle of a file or even in the middle of a function

that library can create new packages

(defun foo ()
  (load-library "https://wisp.town/lib/xyz")
  (xyz:start-xyz)
  #(in-package #:xyz)
  (quit-xyz))

should this work? seems kind of crazy but also kind of reasonable

[mbrock]

maybe rather it's bad that we're currently dropping the whole concept of top-level forms from the server code by wrapping everything in a single (async ...) form

instead we should consider having implicitly async top-level forms

how would that work in the current server.js for example?

we would have to write that in Lisp, I think

(async
  (while (form (read))
    (eval form)))

something like that

[mbrock]

basically done with c57dcf0