IMD-GHOST/Casper testing lang

[djrtwo]

Issue

This testing lang is up for discussion https://notes.ethereum.org/6ozACHyKR9i86vfkt086Ow#.

EDIT: Proposed YAML format: https://notes.ethereum.org/s/r11GVSBuQ

This issue can either be implemented in conjunction with https://github.com/ethereum/beacon_chain/issues/13 or after

[terencechain]

@prestonvanloon has put together the following thoughts. Can you guys take a look and let us know what you think? Thanks!

[mratsim]

Properties

Here are the properties I would find useful:

• Comment support
  • This is important for self documentation, and also quickly disabling/re-enabling tests
  • Note that JSON does not support comment
• Readable
  • This means white-space are insignificant so we can prettify alignment by using whitespaces
• Can be autogenerated
  • This will be helpful to create new test cases
  • Can serve as dump format for logs or crashes for reproduction.
• A clear distinction between input, output and exceptions
  • The exceptions expected should be properly defined as well (i.e. say integer overflow vs array indexing error)

Comments on current proposal

It seems like inspired by CSV for input, with : to separate constants: votes: result, rules should be clear on:

• separator allowed (only , and :) if ever needed:
  • quote ' "
  • escaping quotes, comments, line breaks and the escape characters

What is needed?

To clarify what is needed, we should probably make sure of what the lang should encode, I note:

• Validator
• Epoch length
• Shard Count
• Min Committee Size
• numerical range of validator
• uppercase alphabetical range of fork choice. I probably missed it but are there situation with more than 26 choices?
• specifying empty slots
• validators from multiple sets voting for the same fork 2-7AC [2]6-7AC
• asserting the slot you are in [=6]
• Am I missing something?

Canonical test

This vote part for slot 2 and 3 is a bit unclear 0-5AB, 2-7AC [2]6-7AC

• In slot 2, validators 0-5 of the validator set of slot 2 vote on B, a child of A
• In slot 3, validators 2-7 of the validator set of slot 3 vote on C, a child of A, and validators 6-7 from the validator set of slot 2 do the same

So we could use the following as an equivalent right? 6-7AC [3]2-7AC, [2]0-5AB

[ChihChengLiang]

@mratsim I guess the comma separates what event should happen in when. If we focus on the validators 6-7 defined in slot 2.

• in ..., 0-5AB, 2-7AC [2]6-7AC case: they vote on slot 3 (the 16~23 seconds)
• in ..., 6-7AC [3]2-7AC, [2]0-5AB case: they vote on slot 2 (the 8~15 seconds)

By the way, the [3]2-7AC might be causally problematic, as the validators defined in slot 3 is undefined in slot 2.