search

Interlisp / medley

The main repo for the Medley Interlisp project. Wiki, Issues are here. Other repositories include maiko (the VM implementation) and Interlisp.github.io (web site sources)
https://Interlisp.org
MIT License
376 stars 19 forks source link

Put packages in the WHEREIS database #753

Open rmkaplan opened 2 years ago

rmkaplan commented 2 years ago

You can't do much with a file if its define-file-info has a package that isn't loaded, and the expression itself doesn't say which file to load.

If WHEREIS notices and indexes packages, then DEFINE-FILE-INFO could fetch it

rmkaplan commented 2 years ago

I think this surfaces a more general issue. As far as I can tell, packages are not known as a type on managed files. There is no PACKAGES command in FILEPKG, so no way of knowing where a package comes from, even from a loaded file. It appears that packages are dumped by (P (DEFPACKAGE ...)).

And it is also the case that a managed file has information (like changes) that can't be interpreted without the package, even if the package is defined on that particular file. And a generic tool like FINDCALLERS will break if it comes across a file whose define-file-info mentions on as yet unknown package. The kludge, in a couple of places, is to put the package definition in the define-file-info (this is what the Lyric release notes recommend), or to have the define-file-info go off and load another file to get the needed package.

So here's a proposal:

Make packages be a FILPKGTYPE and add a PACKAGES command to dump restoring expressions for items of that type. It puts out the DEFPACKAGE form that recreates the package as it exists in memory (I hope that that there is a way of avoiding the dump of all currently known symbols or other information that gets incrementally added from other computations).

But also, that command would put out specific marker strings ("Here comes a package" "There went a package") like the EXPORTS command, so that FILEPOS can be used to find just the package definitions on a file in a quick scan, without knowing, loading, noticing anything else.

Then add a function (LOADPKG PACKAGE FILE ) that finds and installs PACKAGE on FILE (by scanning for the marker strings).

Finally, extend DEFINE-FILE-INFO so if a specified package doesn't yet exist, it defaults to calling LOADPKG on itself and the package. Or (LOADPKG package (WHEREIS package 'PACKAGE T)) if the first one fails (and the whereis database tracks packages). [LOADPKG could could also do the WHEREIS internally, as a fallback].

The define-file-info could also just specify (LOADPKG 'anotherfile 'package), and the package would arrive when the expression is evaluated in the ordinary way.

The goal of this is to make it easier to manage packages in a managed-file world in a way that doesn't undermine the other managed-file advantages.

masinter commented 1 year ago

The kludge, in a couple of places, is to put the package definition in the define-file-info (this is what the Lyric release notes recommend), or to have the define-file-info go off and load another file to get the needed package.

I don't understand what's wrong with the "kludge". My impression is that dealing with package declarations is a problem for all Common Lisp implementations.

rmkaplan commented 5 months ago

This is an old issue of relevance to the current discussion of packages. Note that if packages are defined as a filepkgtype, then LOADDEF, GETDEF, WHEREIS... would manipulate packages like everything else. The string search would get package definitions out of their obscure position in the define-file-info, and permit packages to be retrieved and installed from managed files before any of the symbols in that file (or any other file that is the natural place for the package definition) need to be interpreted. It breaks what in linguistics would be called an ordering paradox, where you can't get to the thing you need without having first gotten to it.