`project.janet` can't be eval'ed without access to functions defined in the jpm script

[subsetpark]

The project.janet file is the de facto single destination for project config and metdata, due to its use by jpm. This, plus its inherently open nature as a mostly-normal Janet module, makes it a natural choice for other tools which want to do project management to rely on project.janet as a destination for their own configuration values as well (as well as performing reflection on the project rules already defined in project.janet).

However, this is currently a little difficult to do because project.janet can't simply be required. It needs to be called with require-jpm which defines the macros like declare-project and declare-source. Without those, we can't actually eval the module, we can only parse it.

While it makes sense that the actually evaluation of the jpm rules should be kept inside the jpm script itself, there is therefore some value in making it possible for applications (the one that brings this up for me is Documentarian) to be able to evaluate project.janet to the extent of getting to the data that it expresses, ie, the rules. This would open up the opportunity for the entire ecosystem to use that file as its source of project-specific config.

[andrewchambers]

Everything in core has a runtime cost for every program each time it is run, so be sure its not something that can be done with a library.

[subsetpark]

Happy to pull it out into a standalone library too, just might be weird to have it in a "third party" library but also required by the jpm binary.

Also, happy if project.janet were restructured so it didn't need any extra DSL logic. But obviously that's a major breaking change.

Finally, if all three of the above are no good for their own reasons, then maybe we could all agree on some other project.jdn or config.janet or whatever convention that different apps could use.

[pepe]

For me, the idea of having project.janet as the source of metadata for janet-lang/pkgs comes to mind. But I am also not sure why to have the fn in the stdlib?

[subsetpark]

@pepe because right now, only the jpm script can actually eval project.janet. Your own application can only parse it.

[andrewchambers]

something for spork maybe?

[pepe]

@andrewchambers, that was my thought too.

[bakpakin]

Yeah, if anything I would like to start slimming down boot.janet, not adding 1000 lines of code.

That said, I agree that it would be nice if jpm could be used as a normal module and not just a script.

[andrewchambers]

Oh that makes a lot of sense, simply (import jpm).

[subsetpark]

Yeah, to be clear, I'm not suggesting there's any reason to move this stuff into boot.janet itself. The above thought process comes out of the fact that there's no way to use jpm as a module because it's a script and not on the module path. I know that all the "standard modules" (string, array, etc) are imported by default, but I think there's room for a Python-style extended stdlib that needs to be imported, but is made available as a part of the standard distribution.

[subsetpark]

On reflection, it's clear that my issue title was a little prejudicial. I renamed it to make it clear what would constitute a fix.

[yumaikas]

So, I've found a way to access the contents of a project.janet without needing all of the JPM script bits: Hijacking the expander argument to run-context and/or dofile. We do this at powered-by-janet.

(code below for ease of access)

(defn capture-declares [path] 
    (def *capture* @{})
    (defn append-capture [name & arg]
      (update *capture* name (fn [val] (array/push (or val @[]) ;arg))))
    (defn only-captures [form] 
      (when (string/has-prefix? "declare-" (string (form 0)))
        (append-capture (form 0) (slice form 1))
        form)
      nil)
    (dofile path :expander only-captures)
    *capture*)

Basically, by supplying an expander that return nil, you get to read over every top-level form, but not actually evaluate them, which means that you can pick out the ones that are statically defined. This can be defeated if someone wanted to build a painful project.janet, but any other approach would face essentially the same problem.