New language parser - Don't merge just yet

[rumblesan]

Holy Crap it's a new Parser!

This is a new, custom built parser and interpreter for the LCL language as discussed in the language spec google doc here, https://docs.google.com/document/d/16UmUEUOLnS147bM1zxaNh4y8abrX10onackHo09L4O8/edit?usp=sharing

Not everything is working just yet, for a start, I think it's a bit slower than the previous LCL implementation because we're not eval -ing javascript but interpreting a proper AST. There's tweaking to be done there, but that's a more long term issue.

Things that are broken

• Comments after code don't work so a = 3 // a comment will break things
• Strings aren't implemented so the audio is broken. Should be a quick fix though
• No more doOnce stuff. This functionality (or something like it) will be added back in the future though.
• JS object stuff doesn't work
• No arrays or coffee script for i in [1 .. 3]
• Scoping might be funny
• Could still be some edge cases around blocks and newlines behaving weirdly
• Still needs plenty of testing
• Not all the maths functions are available yet. Simple to add them in though

  Things that are different

Most of these will get changed eventually. Just pointing them out so people know for testing

Functions have changed

At the moment function definition seems to be non functional. Will update when it is.

foo = function (a, b) -> a + b

bar = function (c, d) ->
    box c, d
    return c + d

In the single line function the code after the arrow has to be a simple expression. At the moment this does not include calls to primitives like box, rotate or fill. The value of this expression should be returned. In a function with a block body, you can have statements as normal, but if you want to return something from the function you have to use the return key word. Ideally I'd like to remove this, but it makes parsing things much harder atm and I've not wrapped my head around it.

Things that are working

Quite a few of the new features are now up and running.

Times Loops

Times blocks can have a loop variable, and can also use variables instead of numbers.

loop = 10
loop times with count
    rotate count
    box

Scoped Matrix and colour commands

fill red

rotate time / 1000
    fill blue
        box

box

This will give you a rotating blue box and a still red box.

Inlining for Matrix and Colour commands

The scoping also works when matrix and colour functions are called inline, with optional >> symbols available.

fill red
rotate time, 3, 3 fill blue box
rotate time, 5, 5 >> fill blue >> peg
box

Work still to do

There's still plenty of fixing and tweaking needed, don't think it's ready to merge to master yet, but a bit more work and it will be. For the time being, if people could have a look at it, find bugs and discuss language features and the like that would be ace.

I'll probably update this with more info as I remember, and I'll be pushing more commits up to here.

[davidedc]

niiice. I take it that it's too much work to add this as some sort of swap-in? (I mean so that I can merge progressive work?).

[rumblesan]

So i think i'd prefer to get feedback on this, fix the issues and then merge it in to replace the old language. It probably wouldn't be too much work, but essentially it would end up being wasted work if the old compiler was just going to get removed, along with all the stuff written to get them to work side by side.

tbh, I don't think there's too much work to get it back to having the same (or at least comparable) functionality. I've updated all the demos so they are working just fine, need to do the same for the tutorials as well though

[davidedc]

OK I'll run some of the tests from my list and see which ones apply/work, but I don't feel to attempt a migration of the tests to your harness yet, so it's all going to me manual. Let's see what comes out of a first manual run to get an idea.

[rumblesan]

so yeah, the testing is one thing that i want to get sorted. I ended up using node unit because the parser and preprocessor don't really have any frontend components and it was pretty simple to set up. I'm planning to add more tests once I've got my head around jasmine

[rumblesan]

So users can now define functions correctly. though functions still require the word function before the argument list

[rumblesan]

Also after a bit of testing, it seems that the block scoping on colour commands is working fine.

fill green
noFill
    box
rect
fill red
    peg
ball

the above will give you an empty box, a green rect, a red peg and a green ball. which seems correct to me

[noio]

I have checked out the code for this pull and did the install.

It seems to work pretty okay. Still fast enough but I just got a new computer so there's no fair comparing. I should try & see what happens when there are many lines of code (parsing code is the bottleneck, no?).

There were a couple of little weird things.. Like scale \n box gives a pulsing box, but scale pulse \n box doesn't. I guess that some keywords aren't implemented yet?

Anyway, considering that this is a complete overhaul of the "guts" of lcl, it works really well :D

[rumblesan]

Hey, so this now has the old compiler/program runner working in tandem with the new one. If we can get this merged in then that would make things much easier.

There's some other refactoring I want to do around the way that the compiler, program runner and livecodelab core interact with each other, but i think it makes sense to do that once this is merged.

To switch between the two languages, you need to modify the langVersion parameter in the object that's passed to the LiveCodeLabCore class on creation. It probably wouldn't be too much work to have this changeable from a menu option, but that can also come later if need be