Docs for embedding Elvish?

IngwiePhoenix commented 4 months ago

What new feature should Elvish have?

I have long been wanting to build a build system to have an alternative to CMake - and as an excercise in graph building and processing of such. One very important part in just about any software build is the ability to use at least snippets of scripts - and I would like to use Elvish for that.

And thus, I'd like to know how to embed Elvish as a script language for my project - or more precisely, I would like to request this as a feature if it doesn't exist yet! :)

Output of "elvish -version"

0.21.0-dev.0.20240515223629-06979efb9a2a+official

Code of Conduct

[X] I agree to follow Elvish's Code of Conduct.

krader1961 commented 4 months ago

Embedding any code written in Go inside an app written in another language is problematic. Some of the challenges are due to the Go runtime, how types are defined in each language, and memory ownership. It also creates a tight coupling that causes its own problems. There are other challenges such as how to translate Elvish exceptions over the FFI. One relatively recent article that discusses how to do this is https://www.dolthub.com/blog/2023-02-01-embedding-go-in-c/. I can't speak for the project owner but I doubt this will ever be done and certainly not in the near future. There are just too many other enhancements that could be made which would add considerably more value. Rather than attempting to embed Elvish in a different program using FFI it is likely to be simpler to use an RPC mechanism to interact with an Elvish shell.

IngwiePhoenix commented 4 months ago

Oh - I was more talking about something like

import "src.elv.sh/pkg/..."
func main() {
  var context = Elvish.create()
  context.run(...)
}

But I guess I have to dig through the source more and start with cnd/elvish to find that. Still, thanks for the blog post, that was quite interesting!

IngwiePhoenix commented 4 months ago

Dug around a while and came to this:

An Evaler (...pkgs/eval) and a Parser (...pkgs/parser) are needed.
elvish uses this to set up it's Evaler: https://github.com/elves/elvish/blob/cee682c489925b47d8d46f9afdc31acc19557d10/pkg/shell/shell.go#L99-L132
This here illustrates how to best make a Parser: https://github.com/elves/elvish/blob/cee682c489925b47d8d46f9afdc31acc19557d10/pkg/shell/script.go#L48

So, whilst there is no "made for embedding" API, it is actually quite easy to set this up - and as it is mostly contained in structures, it should? be threadsafe.

However, there is no way to "return" a value from the script to the parent, as far as I can tell. That said, there is some interesting stuff here: https://github.com/elves/elvish/blob/cee682c489925b47d8d46f9afdc31acc19557d10/pkg/eval/vals/struct_map.go

So I imagine it would be possible to just bind a function to the Evaler that would act as a "return" to the host by wrapping the actual code - which, in the tool I want to make, wouldn't be a big problem, since most of the scripts run are rather short snippets, so wrapping them isn't a big deal.

Would be neat to have a plain pkgs/embed module that simplifies this a little; might make it myself in fact, this doesn't seem too difficult - at least, at present. o.o

xiaq commented 3 months ago

@IngwiePhoenix Glad that you figured out most the stuff :)

The reason you can't get a return value is a consequence of Elvish's language semantics: there's simply no concept of return values in Elvish. Instead, Elvish has outputs, and to "return" something you'll need to write it out with put (for value outputs) or echo (for byte output) or similar commands and capture it.

I've added a doc that specifically answers this question: https://github.com/elves/elvish/blob/master/docs/elvish-as-library.md (beware that the godoc pages on pkg.go.dev takes a while to refresh). The examples for Evaler.Eval in particular shows how to get "return" values from Elvish code.

IngwiePhoenix commented 3 months ago

So if I want a real return value, I would have to execute the snippet and then make the code set a global value which I can then pick off the context - so, something remotely similiar to $?. Hey, works for me - simple enough. :)

Thanks for the additions!

xiaq commented 3 months ago

So if I want a real return value, I would have to execute the snippet and then make the code set a global value which I can then pick off the context - so, something remotely similiar to $?. Hey, works for me - simple enough. :)

That, or capture the output if you don't need it for other purposes (and like in Elvish itself, you can pass structured values using builtins like put).

Thanks for the additions!

Np!

elves / elvish