Implement `__newindex` Metamethod

[zicklag]

Hey there! We're considering using piccolo in Jumpy and as I was just browsing and playing around I realized that __newindex isn't implemented yet. We'd probably need that as a way to create proxy types that let us hook into variable assignments.

I might be able to implement it myself, and there's still a chance we don't end up using piccolo or we don't get to testing it out just yet, but I figured I'd open an issue to put it on the radar and see if there was anything I should know if I tried to implement it.

[kyren]

I took a look at it how much effort __newindex would take, to see if it was straightforward, and like most things at this stage of the project, there were a few bumps... but nothing major. I went ahead and implemented it in 1398be0da951d09e04493ed9370917e425ff8632.

I would love more contributors to the project but I have backed off openly calling for active help because, well.. this is just in the stage where there are still a lot of decisions to make, and I would rather have a small core set of devs with very aligned goals rather than try to have a ton of contributors and end up trying to design for every possible use. I think what this project needs is not breadth of contributors but depth.. I would love to have one or two more contributors that really wanted to dig deep and help me hash out the design itself, things like better APIs for dealing with stackless sequences or better callback APIs and things like that. I want help not with the stuff that just takes grunt work but with stuff that takes actual thought and answering questions that don't necessarily have obvious answers (or at least I don't have good answers for yet).

I wanted to just give you a rough guideline for what to do for __newindex and let you take a stab at it but I IMMEDIATELY ran into a silly roadblock which was just something that needed more thought, and we could have hashed it out here but.. github PRs are just a pretty slow way to collaborate sometimes. If you and the other devs for Jumpy are interested in using piccolo.. are you interested in being super involved in design and things like that? We can collaborate on this stuff a lot faster over something like Discord, if you're interested in being more involved, and I can tell you what my near term goals are for piccolo in a lot more detail, to see how well that aligns with what you'd want.

[kyren]

Just to clarify a bit more about what I said before, it's not like I have contributors beating down my door or anything I just don't want to ask for help and then not have answers to like design questions. That's sort of what I ran into when I was working on this the first time, there were just too many unknowns.

I also want to design a Lua interpreter specifically for stackless microthreads, I want to be able to load user code and not worry about it, even to the extent of panics or infinite loops, and if that conflicts with some other goal then, well.. I would rather it be very good at being that kind of runtime and not try to be something else. I think the other major design pillar I want is to maximize flexibility over say, being seamless... we should try to be good at the things PUC-Lua is not as good at, maybe even things like serializing entire interpreter states.

[zicklag]

I went ahead and implemented it in https://github.com/triplehex/piccolo/commit/1398be0da951d09e04493ed9370917e425ff8632.

Awesome, thanks!

I would love more contributors to the project but I have backed off openly calling for active help because, well.. this is just in the stage where there are still a lot of decisions to make, and I would rather have a small core set of devs with very aligned goals rather than try to have a ton of contributors and end up trying to design for every possible use.

Yeah, I totally get it. I've been in the situation a couple times when I start a new project and it's hard to allow new contributors initially because every update means some sort of design decision.

I also want to design a Lua interpreter specifically for stackless microthreads, I want to be able to load user code and not worry about it, even to the extent of panics or infinite loops,

For our uses it would be nice if we could run un-trusted code for mods without having to be afraid of them damaging the user's system or reading the user's secrets. This is mostly related to restricting network or fileystem access, probably.

But anyway, that sounds pretty well aligned if I'm understanding right.

we should try to be good at the things PUC-Lua is not as good at, maybe even things like serializing entire interpreter states.

I like this a lot, too. I ended up making a custom ECS just to get snapshot/determinism support needed for networking. While we should be able to get away without it, being able to also snapshot the Lua interpreter, too could be useful.

If you and the other devs for Jumpy are interested in using piccolo.. are you interested in being super involved in design and things like that?

It's a little hard to tell at this point. I'm going to be in the scripting exploration stage for Jumpy soon-ish probably, but I'm not sure how much time I'll have to invest deeply in the design of piccolo.

That said, the last scripting solution I tried, bevy_mod_js_scripting, I ended up investing quite a bit in that, adding a web backend, extra APIs, etc. Ended up not using it anymore due to performance issues, but if piccolo did seem like it was going to be a good option for us to keep using, then there's a pretty good chance I could budget time to invest in what's necessary to make it work well for us.

Our biggest concerns are probably performance, handling untrusted mods, WASM support, and how nice we can make the interface to our game, in order of priority.

• Lua, the language, with custom metatables, seems like it'd be pretty easy to make a nice API, and is popular for modding, so that is very well aligned for us.
• WASM support should be easy, if it's not already supported right? We just need it to be able to compile for wasm32-unknown-unknown.
• As far as untrusted mods, we may be able to sandbox the game using platform-specific techniques anyway, to help on that front, so it might not be necessary in the runtime.
• Performance just needs to be good enough for mods, which means it probably doesn't need to be ultra high performance.
  • If it had good enough performance for the core game, we might be able to move some core items to Lua code, which would be great for us for hot reloadability and contributor friendliness, but if that doesn't work out, it's not a deal breaker.

I also don't have any background in interpreter/compiler/runtime design, so there might be a learning curve there, and I'm not sure if that would limit my helpfulness. I love learning new things, and I'm getting better and better at it with the more topics that I pick up, so it's not like I'd be afraid to learn or get into the weeds of things, it just might take more time for me to pick up on the context than if I had already done this sort of thing.

---

In summary, it seems piccolo is probably well aligned for us, and there's a possibility I'd be able to get into the weeds of the design work, if it ends up working out for Jumpy, and I have some time to get a handle on it.

For communication, I can't get to Discord, but we're running a Revolt instance that I can get to that we could bridge to Discord if that was fine with @erlend-sh and all the content stays "E for Everyone" or close to it.