Redesign the lua vm.

[rbartlensky]

The interpreter has been moved into its own crate. The new vm takes a different approach when executing instructions. Instead of matching the opcode of the instruction, the interpreter calls the function that is associated to the opcode. Other changes include:

• Removal of the Reg structure
• An executable which reads in a lua file, and interprets it
• Some additional unit-tests

[ltratt]

This needs to be, at least, two commits: one that simply splits the crates up; and another with the new functionality. At the moment I can't work out what's moved code and what's new code.

[rbartlensky]

There is no moved code. 90% of the original interpreter code has been rewritten. The main.rs file is mostly the same as the one from the luacompiler crate.

[ltratt]

OK. Next time, though, I really suggest doing a move first, then doing new code. It does make reviewing easier I promise :)

[rbartlensky]

I never knew RefCells are slow! I changed the way instructions are handled now.

[rbartlensky]

Ready for re-review!

[rbartlensky]

Ready for re-review!

[rbartlensky]

Ready for another review

[ltratt]

I don't understand why clone calls clone_box: why not just call that directly? The extra indirection doesn't seem to be of any use.

[rbartlensky]

I don't understand why clone calls clone_box: why not just call that directly? The extra indirection doesn't seem to be of any use.

This indirection is necessary because pub trait LuaValue: Clone does not compile. This is because LuaValue is used as a trait object. I got this idea from here. I had several attempts to remove the indirection, but none of my ideas compiled.

[ltratt]

Hmm, does that mean every LuaValue is boxed, all the time, and we can't unbox things, should we want to?

[rbartlensky]

Hmm, does that mean every LuaValue is boxed, all the time, and we can't unbox things, should we want to?

I never use LuaNil (or the other structs that implement LuaValue) explicitly. I only have a vector of Boxed LuaValues (the registers) which I need to be able to deep copy.

[ltratt]

Can you expand on deep copy? I use the definitions from https://docs.python.org/2/library/copy.html as a guide -- do they match what you mean?

[rbartlensky]

Can you expand on deep copy? I use the definitions from https://docs.python.org/2/library/copy.html as a guide -- do they match what you mean?

Yes, so in this case, I want to copy both the pointer and the content. I don't want to have two boxes that point to the same LuaValue.

[ltratt]

How is shared mutation going to work if you deep copy things? If I have code that looks something like:

x = {}
x.y = 2
z = x
x.y = 3
print(z)

is it going to print 2 or 3?

[rbartlensky]

How is shared mutation going to work if you deep copy things? If I have code that looks something like:

x = {}
x.y = 2
z = x
x.y = 3
print(z)

is it going to print 2 or 3?

Well in this case, I think I will introduce a LuaAddress. This means that both z and x will contain the same address in their respective registers, so they will reference the same object/dict. Does this make sense?

[ltratt]

OK, so now we're starting to ask an interesting question which, in essence, is "how do we deal with shared mutability?" which implicitly also means "how do we do garbage collection?" @jacob-hughes is working on a Rust GC for this purpose, but it isn't ready yet. In the meantime I would probably use https://crates.io/crates/gc (or perhaps https://github.com/withoutboats/shifgrethor, but my first impression was that it has too odd an API for us to want to use).

Then I would probably have something like:

struct LuaVal {
    val: Gc<LuaObj>
}

enum LuaValKind { INT, FLOAT, GCBOX };

impl LuaVal {
    fn kind(&self) -> LuaValKind { LuaValKind::GCBOX }
    fn to_usize(&self) -> usize { self.val.to_usize() }
    fn gcbox(&self) -> &Gc<LuaObj> { self.val }
    fn get_attr(&self, n: &str) -> Result<LuaVal, ...>
}

trait LuaObj: Gc {
    fn to_usize(&self) -> usize;
    fn get_attr(&self, n: &str) -> Result<LuaVal, ...>
}

struct LuaInt { v: usize }

impl LuaObj for LuaInt {
    fn to_usize(&self) -> usize { self.v }
    fn get_attr(&self) -> Result<LuaVal, ...> { Err("ints have no attributes") }
}

// and so on for each type

why have I split apart LuaVal and LuaObj? The crucial feature here is that LuaVal is opaque to outsiders: val must be entirely private. This will then allow us, in the future, to have unboxed int/floats. In essence, we'd change LuaVal to store a single machine word, and we'd do pointer tagging to determine if it's a GCBOX or an INT etc.; methods like to_usize would then manipulate the tagged pointer/int (when possible). We would only ever store LuaVals (but LuaVals can, of course, point to LuaObjs).

As the above snippet suggest, at first I don't think I'd bother actually implementing the efficiency hack of pointer tagging, but at least the above leaves room for it, while also coping nicely (I think...) with shared mutability.

This is probably too much for this PR, however. So what I'm going to suggest is that you a) squash this PR down b) in a new branch immediately make a test case which errors in the face of shared mutability (i.e. showing that this PR can't implement Lua properly) c) implement something vaguely, roughly along the lines I'm suggesting. Sound like a plan?

[rbartlensky]

I think I see what you mean. I am not quite sure how I can write a test for shared mutability, as my compiler can only handle arithmetic operations on ints and floats.

[ltratt]

Don't forget that you're not restricted by the concrete syntax your system can cope: you can manually instantiate objects, and see how they behave in different situaitons.

[rbartlensky]

Don't forget that you're not restricted by the concrete syntax your system can cope: you can manually instantiate objects, and see how they behave in different situaitons.

That's a good point! I understand now.

I also squashed.