search

kyren / piccolo

An experimental stackless Lua VM implemented in pure Rust
Creative Commons Zero v1.0 Universal
1.68k stars 63 forks source link

This project is no longer on hold! #25

Closed kyren closed 1 year ago

kyren commented 4 years ago

Been going through my open source projects which I've been neglecting and figured I should write something here about where I am with this project.

Unfortunately I'm no longer working on a larger project where I think this might be useful (for my current needs wasm is a better fit), so I'm not going to update this project at least in the near term. I still think the core part of this project (safe interaction with a garbage collector via a futures-ish API) was a neat idea, and I'd still like to explore this in the future when the hopefully the compiler is a bit more ready.

Before I pick this up again though I think that there needs to be additional support in the Rust compiler for doing this, because while it is possible to use a GC safely via combinators, it is not at all pleasant. What I'd like is for it to be possible to have a safe GC'd API using async / await or generators, but AFAICT right now it is not possible. I tried for a while recently to see if I could come up with even a very limited version of the "sequence" API that worked with async / await functions and I couldn't find anything that worked.

You can't auto-generate Collect for closures or generators right now which is certainly a limitiation, but it isn't actually the biggest problem currently. Right now there's no way I can find to pass a Context<'gc> with a unique, branded 'gc lifetime through an async function while having that async function not also be of the 'gc lifetime. We need the async function to strictly outlive 'gc so that it can't hold Gc pointers across await points, or we need some other solution. (Incidentally I can't make this work at all right now, but even if I could I know that you can't make the lifetimes work out so that the async function lives for longer than 'gc).

Once I find any way of proceeding that enables generators-like functions instead of combinators, I think I'll pick this back up, but until then I'm not going to update this.

I'll keep thinking about the problem though! If anybody else has any ideas about how to make this work, let me know!

mkpankov commented 4 years ago

That's a pity, really.

Good luck with your other endeavors!

erlend-sh commented 2 years ago

Hope you don’t mind a bump with a slightly naive question:

Hypothetically speaking would a Rust rewrite of Luau be more feasible?

I figured since Luau is derived directly from Lua whilst also being a complete reimplementation for performance as a game scripting lang, it’s possible that it doesn’t run into the same roadblocks as a Lua VM.

kyren commented 1 year ago

To HECK with it, I miss Lua, and I'm picking this project back up.

Recent gc-arena updates have made some stuff possible that I didn't have a solution to before, like safe, garbage collected userdata.

I still hate Sequence and I still haven't found a way to make that much better (I tried), but I think I can just deal with it.

Hypothetically speaking would a Rust rewrite of Luau be more feasible?

(Answering your question very, very late!)

From quickly looking at the source code, Luau very much seems like a PUC-Rio Lua fork? I'm sure it's probably awesome, and I'm sure Roblox has dramatically improved the safety of vanilla PUC-Rio Lua, just given how much money and engineering talent they have.

The internals of this project have very little to do with how Luau, or any C interpreter derived from PUC-Rio Lua is written. Just as an example, this is in the first file I opened looking through the Luau source code: https://github.com/Roblox/luau/blob/e8c05505867847fbebd8c856e82fe71039c48ccb/VM/src/lvmexecute.cpp#L26 This is just kind of how the Lua VM is written, everything is unsafe everywhere all the time, it's not even "just" normal C unsafety, you need to keep facts about the garbage collector, the Lua stack, and weird C stack tricks (setjmp / longjmp) in mind at all times, for every single line of code. This project is written in an entirely different style, with a safe Sequence based "trampoline" interpreter, and "reified stacks". It might have more in common with something like stackless python than most Lua interpreters?

BUT, again, I'm not trying to compete with Roblox lol, if I had a project that needed a safe Lua VM now, that's the project I would probably use. But nah, I don't really think Luau's existence makes anything more or less feasible for this project in particular.