Support embedded python blocks

[dbarnett]

@tarruda proposed in #29 that vroom syntax should support some kind of embedded python blocks (or python-based DSL) to enable more complex interactions and assertions. This could have a magic vim object defined to interact with vim state.

For example, here's how we could write python code that tests a plugin that highlights in red all capitalized words:

vim.push_keys('iSome word\nanother Word') # Insert some text
vim.command('InvokePlugin') # After this, all capitalized words should be highlighted in red
line1 = vim.screen.windows[0].lines[0] 
line2 = vim.screen.windows[0].lines[1]
assert line1.text == 'Some word'
assert line2.text == 'another Word'
assert line1.attributes[0]['type'] == 'Foreground'
assert line1.attributes[0]['value'] == 'Red'
assert deep_equal(line1.attributes[0].positions, [[0, 4]])
assert line2.attributes[0]['type'] == 'Foreground'
assert line2.attributes[0]['value'] == 'Red'
assert deep_equal(line2.attributes[0].positions, [[8, 12]])

The syntax could look something like:

This is a vroom test

  > iSome Word<esc>o
  > ianother Word<esc>
  > :InvokePlugin<cr>
  >>>

The '>>>' token escapes into a scripting DSL that provides direct access to the API.
We use it to perform more complex assertions. It is still possible to write vroom comments here
  line1 = screen.windows[0].lines[0]
  line2 = screen.windows[0].lines[1]
These statements are translated into python assertions
  line1.foreground should be 'red' from 0 to 4
  line2.foreground should be 'red' from 8 to 12 
The '<<<' token goes back
  <<< 

[dbarnett]

As far as portability, it's probably just as simple to use plain python and provide a common python vim interface instead of creating a DSL. You can even do something like this already with the :python command and the built-in vim module (although I noticed AssertionErrors aren't being handled properly: #39).

[tarruda]

As far as portability, it's probably just as simple to use plain python and provide a common python vim interface instead of creating a DSL. You can even do something like this already with the :python command and the built-in vim module (although I noticed AssertionErrors aren't being handled properly: #39).

If you don't mind, I'd like to give a shot at implementing this DSL for vroom. Even though embedding python is a simpler solution, it would be great to have a DSL that understands statements like line2.foreground should be 'red' from 8 to 12. This also has the advantage of keeping vroom specs language-agnostic

[dbarnett]

Sure, let's get a prototype going and see how it feels.

My biggest concern is just the complexity of having too many different ways to do things in vroom. That would depend on the scope of the DSL, how much documentation it would want, and how much overlap it has with other approaches.

Strong selling points might be if:

• It's way more concise/expressive but conceptually simple
• It unlocks completely new power or flexibility in vroom

For instance, we've discussed different approaches to making assertions (#18). If this just replaced existing approaches as the de facto way to write assertions (maybe with a single-line syntax instead of block syntax), that might work nicely. You might have more high-level information about what's being asserted and be able to give better failure messages.

A few random suggestions:

• I might lean towards not allowing vroom comments inside these blocks, as that makes it feel more like a heavyweight "mode switch" instead of a lightweight block syntax.
• If there are lots of special variables besides "screen", I might suggest still attaching them to a single special namespace or having some mnemonic so the data flow is more obvious.
• We have syntax highlighting implemented in https://github.com/google/vim-ft.vroom (not perfect, but pretty decent). This DSL sounds like it would entail changes there (if nothing more than disabling all highlighting within these blocks). Just be aware that could get ugly for people using the syntax highlighting.

[dbarnett]

Do macros solve this, now that you've added support for those in #46? You should be able to do something like @do (assert_foreground_color, start=8, end=12, color='red') to get such a check, which is about as expressive and readable as a DSL would be.