Progress on field definitions

rikkertkoppes commented 11 years ago

I updated the field definitions to match Robs xml a bit more, see https://github.com/FirstLegoLeague/fllscoring/blob/f8314768fae88b98b4d70b347fdc61e22bb9c593/src/js/views/scores.js

Things work in the work in progress

Key changes:

renamed observables to objectives and nested them in missions. These are now aligned with Robs 'state' part
added constraints, but named them expectations. We need to discuss this a bit more. Rob defines constraints that define invalid combinations (chairTable and chairBase both true) and constraints that define required dependencies (if ballcount==7, ballMiddle must be none). I created a list of expectation functions that must return all true, otherwise an error is presented. This is done for woodworking and strength missions.
removed mission titles from rules as they are in mission definitions now, also removed error throws as they are in the constraints now.

Responding to changes in the interface is now a two step process. For every change, all constraints are checked, then all results are calculated. I have not yet linked the two, but the interface now shows error state and result.

Of course, the interface needs work, but the basic elements are now there.

Further things to think about

combine this with some meta-info, like mission imagery

add constraint messages, it may become something like:

expectations: {
  strength: [{
      message: "Chair cannot be in base AND under the table.",
      exp: function(chairFixedInBase,  chairFixedUnderTable) {
          return !(chairFixedInBase && chairFixedUnderTable);
      }
  }]
}

or

expectations: {
  strength: {
      "Chair cannot be in base AND under the table.": function(chairFixedInBase,  chairFixedUnderTable) {
          return !(chairFixedInBase && chairFixedUnderTable);
      }
  }
}

where the latter is a wee bit more concise

Missions, expectations and rules are implicitly joined by the [mission] keys

rikkertkoppes commented 11 years ago

See this screenshot for the result interface. The strength exercise is in error state

robvw commented 11 years ago

There is a really good point in Meetings/7-10-2013 Standards and workpackages.md Can all observations be expressed as enums?

I think the answer is "yes". I would like to keep the observables as they are now; having distinct types for "yesno", "number" and "enum" can be really helpful for generating an easy-to-understand scoring form. But for the scores, everything would be explicitly treated as an enum. So, instead of this:

    <score type="counter" objective="yellowloops"  points="20"     />
    <score type="yesno"   objective="meds"         yes="25" no="0" />

We'd use this:

    <score objective="yellowloops">
        <item value="0" points="0"  />
        <item value="1" points="20" />
        <item value="2" points="40" />
    </score>
    <score objective="meds">
        <item value="no"  points="0"  />
        <item value="yes" points="25" />
    </score>

For reference, this is the format used for an enum (with "type="enum"" removed, since it's no longer needed).

    <score objective="strength">
        <item value="low"  points="15" />
        <item value="high" points="25" />
        <item value="none" points="0"  />
    </score>

Admittedly it is less concise, but it is much more uniform. Reducing the number of cases makes the format easier to work with, both for a parser and for a person. Note that most scoring rules don't become overly verbose; most rules are of type "yesno", counters with very small numbers or enumerations with just a few possible values. The one big exception is the viruses in the wash basin or the dial with very many possible positions. But in those cases, there usually is no nice formula to go from observable position to number of points; even the official rules simply print a full lookup table. I admit this is ugly:

    <score objective="dial">
        <item value="6" points="-60" />
        <item value="7" points="-55" />
        <item value="8" points="-50" />
        ..
        ..
        <item value="52" points="116" />
        <item value="53" points="117" />
        <item value="54" points="118" />
    </score>

But it is clear and doesn't need ugly hacks to make the formula work out.

Does anybody have further thoughts on this?

kmeesters commented 11 years ago

I think you are right rob. By defining all observations as enums it will also make it easier to create other applications such as a \'challenge generator' or score-sheet generator.

One thing that could be added, when defining the XML and the observations as enums is to perhaps include a a display tag. For example (not an xml expert here).Taking this one step further: we could also include field descriptions

''

And then at the contraints level define indeed the mesages that could be displayed in the app.

note: a final thing: there is an emerging trend that the missions (or better, observables) become intedepent. Looking at this missions from this year, there are depencies between where objects are on the mat, if they are in the same region and so on.

rikkertkoppes commented 11 years ago

@robvw I will change the js file accordingly to use enums

However, we need to think a bit about the way things are displayed in an interface. A dropdown listing all possible numbers of the dial would be inpractical I think.

Can we break it down into two objectives? And have them combined with a lookup table (matrix?) in a rule?

@kmeesters I think something got lost in markdown translation. Can you amend your post?

robvw commented 11 years ago

Maybe I didn't explain clearly. I propose to:

Keep the observables as distinct types (yesno, number, enum)

For observables this gives "hints" which can be used to create the scoring form (yesno and enum: separate buttons, number: slider). This would even allow to treat small numbers (such as "0, 1 or 2") as enums, thus causing them to be drawn differently on the scoring form.
Make all scoring into enum

"Easier" file format.

I'm not decided yet on the dial. One idea is to invent "derived observables": combining multiple observables into a new one according to a formula, thus giving the scoring part values which are more straightforward to work with.

rikkertkoppes commented 11 years ago

those "derived observables" sound a lot like my expressions

I would avoid having observables depend on eachother in order to avoid definition loops. We can define multiple observables per mission already.

Maybe something in the line of

<score mission="strength">
    <case strength="low" points="15" />
    <case strength="high" points="25" />
    <case strength="none" points="0" />
    ...
</score>

<score mission="cardio">
    <case dialbig="1" dialsmall="0" points="-60" />
    <case dialbig="1" dialsmall="1" points="-55" />
    <case dialbig="1" dialsmall="2" points="-50" />
    <case dialbig="1" dialsmall="3" points="-45" />
    ...
</score>

these "variable" attributes are a bit strange in xml. It may be written somewhat different:

<case points="-60">
    <objective name="dialbig" value="1">
    <objective name="dialsmall" value="0">
</case>
...

rikkertkoppes commented 11 years ago

This would translate to my rules functions as follows:

"strength": function(strength) {
    if (strength=='low') {return 15;}
    if (strength=='high') {return 25;}
    if (strength=='none') {return 0;}
},
"cardio": function(dialbig,dialsmall) {
    if (dialbig==1 && dialsmall==0) {return -60;}
    if (dialbig==1 && dialsmall==1) {return -55;}
    if (dialbig==1 && dialsmall==2) {return -50;}
    if (dialbig==1 && dialsmall==3) {return -45;}
}

robvw commented 11 years ago

First things first: I'm not worried about representing it in JS, that shouldn't be a problem, it''s defining a good way to represent things in XML which is the challenge in my opinion.

So essentially we need a multi-valued attribute to determine which observables a score depends on. How about something like this:

<score>
    <objective>dialbig</objective>
    <objective>dialsmall</objective>
    <case dialbig="1" dialsmall="0" points="-60" />
    <case dialbig="1" dialsmall="1" points="-55" />
    <case dialbig="1" dialsmall="2" points="-50" />
    ..
    ..
    <case dialbig="8" dialsmall="4" points="116" />
    <case dialbig="8" dialsmall="5" points="117" />
    <case dialbig="9" dialsmall="0" points="118" />
</score>

Technically, it should probably be:

    <case dialbig="1" dialsmall="0">-60</case>

But I feel that's harder to read.

An alternative, but kinda ugly, solution would be to tacitly assume all scores depends on all observables; when parsing the XML, look for all of them (any observables which are not defined automatically match). This would allow stuff like:

<score>
    <case a="1" b="1" points="10" />
    <case a="1" b="2" points="25" />
    <case a="2"       points="50" />
</score>

(In the case a=2, the value of b doesn't matter; leaving it out is nicer than forcing two lines which are identical except for the value of b.) No existing missions which work like that come to mind, but it might be good future-proofing.

Also, Kenny has a good point. One solution would be to run through all possible combinations (daddy in green zone + mommy in green zone + kid in green zone = x points; daddy in blue zone + mommy in blue zone + kid in blue zone = x points; etc...), but the number of combinations would probably explode (not to mention it being very sensitive to bugs / omissions)... And in the XML I can't exactly do something like:

    <case daddy="mommy" points="x" />

rikkertkoppes commented 11 years ago

Wondering, if we list all possible combinations anyway, the constraints can go away right?

rikkertkoppes commented 11 years ago

   <score>
       <case a="1" b="1" points="10" />
       <case a="1" b="2" points="25" />
       <case a="2"       points="50" />
   </score>

Looks good, but add the a reference to a mission: <score mission="foo">

Adding formula's / expressions to the xml brings us back to square one: it is hard / impossible to write expressions that are easily parsed in implementations.

rikkertkoppes commented 11 years ago

There is http://www.w3.org/TR/xexpr/ but it seems quite verbose

robvw commented 11 years ago

Looks good, but add the a reference to a mission:

Do you really mean a mission, or did you mean an observable?

In the first case, why? It's just internal stuff, the user won't see it anyway.

In the second case, the "a" and "b" are the observables; I didn't use real names because no examples came to mind.

Adding formula's / expressions to the xml brings us back to square one: it is hard / impossible to write expressions that are easily parsed in implementations.

Indeed... but one way or another we will need to come up with a complete description. One of this year's missions is for example:

WATER Conditions visible at the end of the match: — At least one person is together with (bottled) water in the same region. Value: PEOPLE WITH AT LEAST 1 WATER…15 EACH

(source)

I haven't seen the scoring form yet, but the probably options are:

For each area a question "How many people are here?" and "How many water bottles are here?"
For each person and each water bottle a question ""In which area is this person / water bottle?"

In either case it's difficult to match things up without any kind of formula (if memory serves, there's three people, two water bottles and half a dozen or so areas; to make matters worse, there's similar missions for pets and tools as well...).

Wondering, if we list all possible combinations anyway, the constraints can go away right?

Nope, there's a huge difference between "no score" and "that can't happen; referee, fix this!".

rikkertkoppes commented 11 years ago

What about using haxe (http://haxe.org/) as the expression language? It would allow others to have the parts compiled to their language of choice. Indeed for missions such as WATER things can explode really quickly.

To be bold: maybe write the entire field definition in haxe? And generate XML from that. Any application that uses the XML would probably not use the expressions at all.

Let's park the constraints discussion for now to not have two discussions running simultaneously.

I really meant mission in the score element, to be able to couple a score back to a mission to be able to show that in the interface.

kmeesters commented 11 years ago

Just my 2 cents: but employing another language at the 'parent' would defeat the reason why we went with XML in the first place. If we generate the XML from Haxe, then users become dependent on our files, (I assume that the Haxe cannot be created from XML). So that means that Users cannot use our system unless we provide them with the Haxe files, inclduing translation issues and so on. Otherwise we could have gone with Javascript definitions to begin with.

The technical difficulties aside, I would really push for XML, even if compromising at some points. XML as the master file would enable quicker worldwide adaption and other uses. We could provide different adapters (parsers) along with the XML. Again, just my 2 cents.

rikkertkoppes commented 11 years ago

The thing with haxe is that it can be compiled to a lot of languages. We could do that and publish the same field definition files in all those languages (js, java, php, c++, c# etc). Hence, it would still be available to a wide audience.

kmeesters commented 11 years ago

My point is, what happens is people (users/partners) modify the XML, because this is the most 'readable' to them. For example imagine the partner from Barsil translating the missions into Portoguese.

rikkertkoppes commented 11 years ago

Translation is not a problem anyway, in either case descriptions and such can be extracted from the source file, being it XML or any other language. It is not a factor in the discussion.

rikkertkoppes commented 11 years ago

I updated field definitions (JS version) to include all missions from the 2012 FLL: https://github.com/FirstLegoLeague/fllscoring/blob/37e757b47857a1f6ecb29ad585813b6e4988cb9f/src/js/views/scores.js

Highlights:

added an extra "mission" to include team color observables.
made it possible for mission rules to depend on observables outside the mission (for ball game)
the cardio mission rule is pretty straightforward now (a lookup table), but the expectations are a bit more cumbersome. This is still a bit of logic, hard to define in XML.

The interface now looks like this (note team color feedback and error feedback and ball game dependency on team color). It needs a lot of love still

robvw commented 11 years ago

Weird question maybe, but you only use checkboxes and textboxes (with updown arrows). For some missions it would make a lot of sense to use radiobuttons (those round ones, where only one can be selected at a time; a great match for enumerations). Is there a special reason you don't (can't) use these?

I'm kinda low on time at the moment; next update to XML proposal will probably be next Tuesday or Wednesday.

rikkertkoppes commented 11 years ago

Main reason is I did not yet add enum types to my definitions. My efforts are mainly exploratory, an exercise to see what I come across. Team color dependency is one thing.

I'll try to come up with something for enums in the next few days. They'll save a few constraints as well.

rikkertkoppes commented 11 years ago

I was thinking to define enums as follows:

"Chair": {
    "type": Enum, 
    "title": ["chair fixed in base","chair fixed under table"]
}

Would that fit your ideas?

rikkertkoppes commented 11 years ago

Some more updates: https://github.com/FirstLegoLeague/fllscoring/blob/843ecdab614470a1921567a2e3c54e8fa4475fff/src/js/views/scores.js

I introduced Enums, which make stuff a lot easier. Furthermore, since expectations (constraints) and scores are tied to missions anyway, I shoved them under the same definition. In xml equivalent:

<fll>
    <details>
        <season year="2012" name="Senior Solutions" />
        # Anything else?
    </details>

    <state>
        <mission description="Wood work">
            <objective id="chairbase" description="" type="yesno" />
            <objective id="chairtable" description="" type="yesno" />
            <constraint id="chairlocation" type="invalid" value="1" direction="max" message="Chair cannot be in base AND under the table.">
                <item id="chairbase" value="yes" />
                <item id="chairtable" value="yes" />
            </constraint>
            <score type="yesno"   objective="chairbase"    yes="15" no="0" />
            <score type="yesno"   objective="chairtable"   yes="25" no="0" />
        </mission>
    </state>
</fll>

Or something similar (we might group scores and constraints)

The transition to enums simplifies as follows:

"woodworking":{
    "title":"Wood Working",
    "objectives": {
        "chairFixedInBase":{
            "title":"Chair fixed in base",
            "type":"Boolean"
        },
        "chairFixedUnderTable":{
            "title":"Chair fixed and any part under table",
            "type":"Boolean"
        }
    },
    "expectations":[
        function(chairFixedInBase,  chairFixedUnderTable) {
            return !(chairFixedInBase && chairFixedUnderTable);
        }
    ],
    "score":function(chairFixedInBase,chairFixedUnderTable) {
        return chairFixedInBase*15 + chairFixedUnderTable*25;
    }
},

To:

"woodworking":{
    "title":"Wood Working",
    "objectives": {
        "chairPosition": {
            "title": ["Chair fixed in base","Chair fixed and any part under table","Chair elsewhere"],
            "type": "Enum"
        },
    },
    "score":function(chairPosition) {
        return [15,25,0][chairPosition];
    }
},

rikkertkoppes commented 11 years ago

I propose to move the descriptions of the objectives to option child elements, this way, we can align boolean, number and enum types a bit more:

<mission description="Quiting">
    <objective id="blueQuilts" type="Number" max="2">
        <option description="Blue quilt square touching their black target regions"/>
    </objective>
    <objective id="orangeQuilts" type="Number" max="2">
        <option description="Orange quilt square touching their black target regions"/>
    </objective>
</mission>
<mission description="Wood Working">
    <objective id="chairPosition" type="Enum">
        <option description="Chair fixed in base"/>
        <option description="Chair fixed and any part under table"/>
        <option description="Chair elsewhere"/>
    </objective>
</mission>
<mission description="Medicine">
    <objective id="orangeBottleMoved" type="Boolean">
        <option description="Orange bottle moved or outside lines"/>
    </objective>
    <objective id="greenBottleInBase" type="Boolean">
        <option description="Green bottle in base"/>
    </objective>
</mission>

rikkertkoppes commented 11 years ago

Also, Kenny has a good point. One solution would be to run through all possible combinations (daddy in green zone + mommy in green zone + kid in green zone = x points; daddy in blue zone + mommy in blue zone + kid in blue zone = x points; etc...), but the number of combinations would probably explode (not to mention it being very sensitive to bugs / omissions)... And in the XML I can't exactly do something like:
<case daddy="mommy" points="x" />

@kmeesters can we circumvent these problems by rephrasing the question (can we rephrase questions?), like:

<objective id="familyPosition" type="Enum">
    <option description="Dad, mom, kid in green zone"/>
    <option description="Dad, mom, kid in blue zone"/>    
</objective>
<score>
    <case familyPosition="0" points="30" />
    <case familyPosition="1" points="50" />
<score>

Lastly, if we treat everything as enums, we can list all possible scores with cases as the above. The xml may get lengthy and options may explode, but it will work. The biggest definition I've seen is the cardio mission of last year (with the dials).

For example, the "family" mission of this year (seems most complicated) could be defined as follows:

<mission description="Family">
    <objective id="largestGroup" type="Number" max="3">
        <option description="People in largest group in a single Region"/>
    </objective>
    <objective id="peopleWithWater" type="Number" max="3">
        <option description="People in Regions containing bottled water"/>
    </objective>
    <objective id="peopleInYellow" type="Number" max="3">
        <option description="People in the Yellow Region"/>
    </objective>
    <objective id="peopleInRed" type="Number" max="3">
        <option description="People in the Red Region"/>
    </objective>
    <objective id="petsWithPeople" type="Number" max="2">
        <option description="Pets in Regions containing People"/>
    </objective>
    <score>
        <case largestGroup="0" points="0" />
        <case largestGroup="1" points="0" />
        <case largestGroup="2" points="33" />
        <case largestGroup="3" points="66" />
    <score>
    <score>
        <case peopleWithWater="0" points="0" />
        <case peopleWithWater="1" points="15" />
        <case peopleWithWater="2" points="30" />
        <case peopleWithWater="3" points="45" />
    </score>
    <score>
        <case peopleInYellow="0" points="0" />
        <case peopleInYellow="1" points="12" />
        <case peopleInYellow="2" points="24" />
        <case peopleInYellow="3" points="36" />
    </score>
    <score>
        <case peopleInRed="0" points="0" />
        <case peopleInRed="1" points="18" />
        <case peopleInRed="2" points="36" />
        <case peopleInRed="3" points="54" />
    </score>
    <score>
        <case petsWithPeople="0" points="0" />
        <case petsWithPeople="1" points="15" />
        <case petsWithPeople="2" points="30" />
    </score>
</mission>

And the Supplies and Equipment mission (long lists) as follows:

<mission description="Supplies and Equipment">
    <objective id="nonWaterInYellow" type="Number" max="12">
        <option description="Non-water items in Yellow Region"/>
    </objective>
    <objective id="nonWaterInRed" type="Number" max="12">
        <option description="Non-water items in Red Region"/>
    </objective>
    <score>
        <case nonWaterInYellow="0" points="0" />
        <case nonWaterInYellow="0" points="3" />
        <case nonWaterInYellow="0" points="6" />
        <case nonWaterInYellow="0" points="9" />
        <case nonWaterInYellow="0" points="12" />
        <case nonWaterInYellow="0" points="15" />
        <case nonWaterInYellow="0" points="18" />
        <case nonWaterInYellow="0" points="21" />
        <case nonWaterInYellow="0" points="24" />
        <case nonWaterInYellow="0" points="27" />
        <case nonWaterInYellow="0" points="30" />
        <case nonWaterInYellow="0" points="33" />
        <case nonWaterInYellow="0" points="36" />
    </score>
    <score>
        <case nonWaterInRed="0" points="0" />
        <case nonWaterInRed="0" points="4" />
        <case nonWaterInRed="0" points="8" />
        <case nonWaterInRed="0" points="12" />
        <case nonWaterInRed="0" points="16" />
        <case nonWaterInRed="0" points="20" />
        <case nonWaterInRed="0" points="24" />
        <case nonWaterInRed="0" points="28" />
        <case nonWaterInRed="0" points="32" />
        <case nonWaterInRed="0" points="36" />
        <case nonWaterInRed="0" points="40" />
        <case nonWaterInRed="0" points="44" />
        <case nonWaterInRed="0" points="48" />
    </score>
</mission>

Both are length, but trivial

Another solution for these trivial lists is something like

<mission description="Supplies and Equipment">
    <objective id="nonWaterInYellow" type="Number" max="12">
        <option description="Non-water items in Yellow Region"/>
    </objective>
    <objective id="nonWaterInRed" type="Number" max="12">
        <option description="Non-water items in Red Region"/>
    </objective>
    <score>
        <mult nonWaterInYellow="3" />
    </score>
    <score>
        <mult nonWaterInRed="4" />
    </score>
</mission>

To define trivial multiplication (the cardio mission would still be non-trivial, requiring the entire list)

kmeesters commented 11 years ago

Wait, I made a good point? That was entirely by accident!.. ehm.. I mean, Of course I know that is why I brought it up...

I think you can go two ways with these combinations. Either break it down to very specific individual misson model observations:

- Mom is the:
   - green zone
  - blue zone
- Dad is in the:
   - green zone
  - blue zone

The use a condition to combine the both:

Mom zone = Dad zone => 25 points

Or you can make the conditions combined, like you mentioned above.

In general, also in regards to the rephrasing question, I think we should keep true to the scoring form. This is the global standard set by first. Introducing different scoring options or phrasing might interfere with referee training (FLL rules and missions are very sensitive to proper wording).

I will create a higher adoption for our system if we can almost copy the scoring sheet from FIRST one-to-one. Attached is a screenshot of the official one. (political reasons). I think it is clean fll 2013 nature s fury accessible score sheet

Here is a sample interpretation which I'm not particually fond of (differs to much in style, wording and options from the offical one): http://www.flltournament.com/Scoresheet.aspx?CID=15

robvw commented 11 years ago

Here's a thoroughly revised version of the XML. Things to note:

A lot of structures have been greatly simplified and made more uniform.
I did opt to go with using objective IDs in key-value pairs. This greatly helps with human-readability but I believe it does not hurt computer-readability (parsability).
The ugly, convoluted way of handling constraints is gone! :) The new way is simple, clear and very similar to the way scores are described.
Error messages are duplicated when they have to appear in multiple situations. I couldn't think of a way to prevent this without adding more complexity to the format (unless using error IDs and a separate list of error messages would help with localisation??). If anybody has a suggestion, I'd be very interested.
I'm unsure whether a score in one mission should be allowed to depend on an objective (observable) defined in another mission... @kmeesters: could you say something about that? (Note, it's not something which would affect parsing all that much, but it is something I want to explain explicitly, one way or the other, in a readme.)
The complete description is roughly 10 kB. That's a little bigger than originally expected, but not something I foresee causing trouble. @rikkertkoppes: is there a better way to share this than posting as a comment.
If everybody agrees this format is acceptable, I'll try to encode the challenge for this year, to test how future-proof it is; once defined I hope we won't need to change it again for a while.

2012 - Senior Solutions.xml

<fll>

    <season year="2012" name="Senior Solutions" />
    <!-- Anything else? -->

    <mission description="Flexibility">
        <objective id="yellowloops" description="Yellow Loops in Base" type="number" min="0" max="2" />
        <!--
            The following could be added, but it'd serve no purpose:
            <score yellowloops="0" points="0"  />
        -->
        <score yellowloops="1" points="20" />
        <score yellowloops="2" points="40" />
    </mission>

    <mission description="Medicine">
        <objective id="meds" description="Green in Base, Orange Not Moved" type="yesno" />
        <score meds="yes" points="25" />
    </mission>

    <mission description="Helping Animals">
        <objective id="animals" description="Dog in Base" type="yesno" />
        <score animals="yes" points="20" />
    </mission>

    <mission description="Wood work">
        <objective id="chairbase"  description="Chare Repaired and in Base"     type="yesno" />
        <objective id="chairtable" description="Chare Repaired and under Table" type="yesno" />
        <constraint chairbase="yes" chairtable="yes" message="Chair cannot be in base AND under the table." />
        <score chairbase="yes"  points="15" />
        <score chairtable="yes" points="25" />
    </mission>

    <mission description="Videophone">
        <objective id="flags" description="Flags Fully Upright" type="number" min="0" max="2" />
        <score flags="1" points="20" />
        <score flags="2" points="40" />
    </mission>

    <mission description="Quilts">
        <objective id="quiltsblue"   description="Blue Squares Touch Target" type="number" min="0" max="2"   />
        <objective id="quiltsorange" description="Orange Squares Touch Target" type="number" min="0" max="2" />
        <score quiltsblue="1"   points="15" />
        <score quiltsblue="2"   points="30" />
        <score quiltsorange="1" points="30" />
        <score quiltsorange="2" points="60" />
    </mission>

    <mission description="Cooperation and Recognising Similarities">
        <objective id="coop" description="Dials on Both Fields are Parallel" type="yesno" />
        <score coop="yes" points="45" />
    </mission>

    <mission description="Ball Game">
        <objective id="ballcount" description="Balls on Rack" type="number" min="0" max="7" />
        <objective id="ballmiddle" description="Middle Ball" type="enum">
            <option id="your" description="Yours" />
            <option id="them" description="Theirs" />
            <option id="none" description="Neither" />
        </objective>
        <constraint ballcount="0" ballmiddle="your" message="If no balls are left, the middle ball must be 'Neither'." />
        <constraint ballcount="0" ballmiddle="them" message="If no balls are left, the middle ball must be 'Neither'." />
        <constraint ballcount="1" ballmiddle="none" message="When some, but not all, balls are left, the middle ball cannot be 'Neither'." />
        <constraint ballcount="2" ballmiddle="none" message="When some, but not all, balls are left, the middle ball cannot be 'Neither'." />
        <constraint ballcount="3" ballmiddle="none" message="When some, but not all, balls are left, the middle ball cannot be 'Neither'." />
        <constraint ballcount="4" ballmiddle="none" message="When some, but not all, balls are left, the middle ball cannot be 'Neither'." />
        <constraint ballcount="5" ballmiddle="none" message="When some, but not all, balls are left, the middle ball cannot be 'Neither'." />
        <constraint ballcount="6" ballmiddle="none" message="When some, but not all, balls are left, the middle ball cannot be 'Neither'." />
        <constraint ballcount="7" ballmiddle="your" message="If all balls are left, the middle ball must be 'Neither'." />
        <constraint ballcount="7" ballmiddle="them" message="If all balls are left, the middle ball must be 'Neither'." />
        <score ballcount="1"     points="10" />
        <score ballcount="2"     points="20" />
        <score ballcount="3"     points="30" />
        <score ballcount="4"     points="40" />
        <score ballcount="5"     points="50" />
        <score ballcount="6"     points="60" />
        <score ballcount="7"     points="70" />
        <score ballmiddle="your" points="60" />
    </mission>

    <mission description="Gardening">
        <objective id="plants" description="Base of Plants Touch Target" type="yesno" />
        <score plants="yes" points="25" />
    </mission>

    <mission description="Stove">
        <objective id="stove" description="All Burners are Black" type="yesno" />
        <score stove="yes" points="25" />
    </mission>

    <mission description="Bowling">
        <objective id="pins" description="Pins Hit" type="number" min="0" max="6" />
        <score pins="1" points="7"  />
        <score pins="2" points="14" />
        <score pins="3" points="21" />
        <score pins="4" points="28" />
        <score pins="5" points="35" />
        <score pins="6" points="60" />
    </mission>

    <mission description="Transitions">
        <objective id="platformslanted" description="Robot Only Touches Slanted Platform" type="yesno" />
        <objective id="platformbalanced" description="Robot Only Touches Balanced Platform" type="yesno" />
        <objective id="platformclear" description="Platform Only Touches Robot and Mat" type="yesno" />
        <constraint platformslanted="yes" platformbalanced="yes" message="Platform cannot be slanted AND balanced." />
        <score platformslanted="yes"  platformclear="yes" points="45" />
        <score platformbalanced="yes" platformclear="yes" points="65" />
    </mission>

    <mission description="Strength Training">
        <objective id="strength" description="Weight raised" type="enum">
            <option id="low" description="Low" />
            <option id="high" description="High" />
            <option id="none" description="Not Done" />
        </objective>
        <score strength="low"  points="15" />
        <score strength="high" points="25" />
    </mission>

    <mission description="Cardio Training">
        <objective id="dialbig" description="Dial Big Step" type="number" min="1" max="9" />
        <objective id="dialsmall" description="Dial Small Step" type="number" min="0" max="5" />
        <constraint dialbig="9" dialsmall="1" message="When Big Dial is on 9, Small Dial can only be on 0." />
        <constraint dialbig="9" dialsmall="2" message="When Big Dial is on 9, Small Dial can only be on 0." />
        <constraint dialbig="9" dialsmall="3" message="When Big Dial is on 9, Small Dial can only be on 0." />
        <constraint dialbig="9" dialsmall="4" message="When Big Dial is on 9, Small Dial can only be on 0." />
        <constraint dialbig="9" dialsmall="5" message="When Big Dial is on 9, Small Dial can only be on 0." />
        <score dialbig="1" dialsmall="0" points="-60" />
        <score dialbig="1" dialsmall="1" points="-55" />
        <score dialbig="1" dialsmall="2" points="-50" />
        <score dialbig="1" dialsmall="3" points="-45" />
        <score dialbig="1" dialsmall="4" points="-40" />
        <score dialbig="1" dialsmall="5" points="-35" />
        <score dialbig="2" dialsmall="0" points="-30" />
        <score dialbig="2" dialsmall="1" points="-25" />
        <score dialbig="2" dialsmall="2" points="-20" />
        <score dialbig="2" dialsmall="3" points="-15" />
        <score dialbig="2" dialsmall="4" points="-10" />
        <score dialbig="2" dialsmall="5" points="-5"  />
        <score dialbig="3" dialsmall="0" points="0"   />
        <score dialbig="3" dialsmall="1" points="5"   />
        <score dialbig="3" dialsmall="2" points="10"  />
        <score dialbig="3" dialsmall="3" points="15"  />
        <score dialbig="3" dialsmall="4" points="20"  />
        <score dialbig="3" dialsmall="5" points="25"  />
        <score dialbig="4" dialsmall="0" points="30"  />
        <score dialbig="4" dialsmall="1" points="35"  />
        <score dialbig="4" dialsmall="2" points="40"  />
        <score dialbig="4" dialsmall="3" points="45"  />
        <score dialbig="4" dialsmall="4" points="50"  />
        <score dialbig="4" dialsmall="5" points="55"  />
        <score dialbig="5" dialsmall="0" points="60"  />
        <score dialbig="5" dialsmall="1" points="63"  />
        <score dialbig="5" dialsmall="2" points="66"  />
        <score dialbig="5" dialsmall="3" points="69"  />
        <score dialbig="5" dialsmall="4" points="72"  />
        <score dialbig="5" dialsmall="5" points="75"  />
        <score dialbig="6" dialsmall="0" points="78"  />
        <score dialbig="6" dialsmall="1" points="91"  />
        <score dialbig="6" dialsmall="2" points="94"  />
        <score dialbig="6" dialsmall="3" points="97"  />
        <score dialbig="6" dialsmall="4" points="100" />
        <score dialbig="6" dialsmall="5" points="103" />
        <score dialbig="7" dialsmall="0" points="106" />
        <score dialbig="7" dialsmall="1" points="107" />
        <score dialbig="7" dialsmall="2" points="108" />
        <score dialbig="7" dialsmall="3" points="109" />
        <score dialbig="7" dialsmall="4" points="110" />
        <score dialbig="7" dialsmall="5" points="111" />
        <score dialbig="8" dialsmall="0" points="112" />
        <score dialbig="8" dialsmall="1" points="113" />
        <score dialbig="8" dialsmall="2" points="114" />
        <score dialbig="8" dialsmall="3" points="115" />
        <score dialbig="8" dialsmall="4" points="116" />
        <score dialbig="8" dialsmall="5" points="117" />
        <score dialbig="9" dialsmall="0" points="118" />
    </mission>

</fll>

rikkertkoppes commented 11 years ago

So we have three types: enum, yesno and number. Only the first take options in the objective to be able to pass the descriptions.

Contains follow the same structure as the scores, but communicate an error message instead of points.

It looks workable, i'll check whether this works with a js version, but I think it should be no problem.

I think you should be able to create a wiki page, that would be a good destination for the spec. Also, do you plan on creating a schema? That would allow us to check definitions for correctness automatically.

robvw commented 11 years ago

I found the wiki, but no way to create (or edit) a page.

Yes, that's the plan; is there any formalism you would prefer? I was thinking about BNF but didn't yet check for tools which can automatically validate.

On 22/10/2013, rikkertkoppes notifications@github.com wrote:

So we have three types: enum, yesno and number. Only the first take options in the objective to be able to pass the descriptions.

Contains follow the same structure as the scores, but communicate an error message instead of points.

It looks workable, i'll check whether this works with a js version, but I think it should be no problem.

I think you should be able to create a wiki page, that would be a good destination for the spec. Also, do you plan on creating a schema? That would allow us to check definitions for correctness automatically.

Reply to this email directly or view it on GitHub: https://github.com/FirstLegoLeague/fllscoring/issues/4#issuecomment-26777801

rikkertkoppes commented 11 years ago

I'll check the wiki.

Use xml schema (xsd) http://www.w3.org/XML/Schema

rikkertkoppes commented 11 years ago

@robvw wiki should be editable now

robvw commented 11 years ago

Riiiight (I obviously don't work with XML enough).

One problem, Schema doesn't seem to allow stuff like

<score platformbalanced="yes" platformclear="yes" points="65" />

all the keys need to be predefined strings, they cannot be declared elsewhere in the XML. I posted a question on stackoverflow but I'm afraid the answer will be we have to go with

<score points="65">
    <objective id="platformbalanced" value="yes" />
    <objective id="platformclear" value="yes" />
<score />

if we want scores to depends on multiple objectives and have validation. It's not the end of the world, but it is still a pity. Should that indeed turn out to be the answer, I want to propose we don't "shortcut" the scores depending on a single objective, because it hurts uniformity.

rikkertkoppes commented 11 years ago

The xsd and final XML files should end up in the repo, or maybe in a separate repo? In the wiki we should have a "human readable" version of the xsd so to say. It is also a great place to store a work in progress of that.

I'd vote for uniformity as well

What are your thoughts about

<score>
    <mult id="nonWaterInRed" points="4" />
</score>

To simplify lists of scores that are just multiplicative?

Oh and by the way, don't reuse the objective element there. That would be confusing

robvw commented 11 years ago

As soon as I have GIT installed I will make a commit and send you a pull request.

It doesn't complicate parsing or validating all that much, so we could certainly add it.

Personally I have a slight preference for not adding it though: it is not a bad idea, but I do think it's (in practice) a superfluous addition. Whenever there's a count which can grow pretty big (two years ago: viruses in sink; last year: position of the dial; this year: shared mission) the points are not a simple multiplication anyway. When numbers don't grow that big (usually the maximum is two or three) this is only a small benefit, which may not be worth the additional effort.

Having said that, it's only a minor preference; if you suggested it because you believe it should be included (difficult to tell from an "What are your thoughts"), sure, we can include it!

Regarding naming: good point. New suggestion: "condition"; that works well for both constraints and scores.

robvw commented 11 years ago

Sorry, forgot to reply to this part:

The xsd and final XML files should end up in the repo, or maybe in a separate repo?

I'd say just put them in the main repository; the main app can't do much without those files, so they should be there. Making a separate repository would make a lot of sense if they changed and evolved a lot on their own, but I kinda want to get them right on the first try and never touch them again.

Making a separate repository for the localisation files (when we get to them) might be an idea... I can work with GIT, but am by no means an expert, so I don't have a good overview of the pros and cons.

kmeesters commented 11 years ago

Regarding the interdependency of the observables.

Missions are always (or at least have always) been independent. The score of one missions is not directly related to the status of another mission. However there have been implicit relations. For example you can only place an object in one area or the other. Or by completing one mission in one way you may not be able to score the other mission.

For example look at the mission of this year. If a team gets a touch penalty, then the runway is not clear or rubble anymore and the plane may not reach the end of the runway (LT blue area). However this iis not explicitly defined in the mission.

Observations are shared in some way, for example, the runway is used in 'clear runway' mission but also (partly) in the position of the ambulance. However by defining the observations different ('ambulance position' instead of 'what is on the yellow area') the can be viewed independent. Take a look at the sample form, somethings may be shared (for examples 'waves') but are recorded separately.

To answers more shortly: so no, the observables are not dependent but the can be 're-used' (based on previous challenges)

robvw commented 11 years ago

Kenny's explanation sounds like scores only depend on observables in their own mission (at least, up to this season's challenge), but it seems plausible constraints sometimes need to refer to observables from another mission.

I'm writing a first version of the XML specs right now; once that's posted to the wiki I'll try to codify this season's challenge and see what happens. More discussion when that's done!

kmeesters commented 11 years ago

Great, just a heads up, this year a few 'observables' are defined in the rules rather then the msisions. This is the sprawl penalty and the junk penalty (I almost missed them, tip for Scott: define point related stuff in missions the rest in rules :) )

robvw commented 11 years ago

This year's Challenge is ehm... well, quite a challenge to encode neatly. Maybe the format does need a few more tweaks.

First of, Rikkert, I changed my mind mind about your proposal. To keep it more in line with the rest of the format, I slightly changed it though, how about this:

<score points="4">
    <condition id="itemsred" value="each" />
</score>

For last year it won't help all that much, but this year it does indeed make life easier.

The scoring sheet has two questions: "Non-water items in Yellow Region" and "Non-water items in Red Region", both wanting an number between 0 and 12 for an answer. Since there's only 12 items in total, that means disallowing a ton of combinations (1/12, 2/12, .., 12/12, 2/11, 3/11, .., 12/11, ..).

Unfortunately I'm not sure how to best solve this, does anybody happen to have a great idea? The best I came up with isn't very elegant:

<constraint sum="12" message="Total number of supplies in yellow and red region cannot exceed 12.">
    <condition id="itemsyellow" value="sum" />
    <condition id="itemsred"    value="sum "/>
</constraint>

In fact, I'm doubting whether to solve it at all; sure, it's ugly to have half your file taken up by a bunch of lame checks... but it does work.

The part which probably does need a solution is repeating the error message over and over again. A typo in one occurrence could mess up localisation and after copy-pasting it a couple dozen times, changing your mind about the wording is not overly fun... The best solution I came up with was "when the message is omitted, reuse the last message". This seems workable but is not a beautiful solution; better ideas would be welcome!

Kenny, I sent a clarification request to Scott asking about those junk penalties. The scoring form only goes up to eight of each, but the rules specify no maximum. Did you already send the message about "tip for Scott: define point related stuff in missions the rest in rules"; just posting it here doesn't do much good after all. :p

robvw commented 11 years ago

ps. Do we want to discuss this here, or is it easier to just talk about it on Monday, face to face, then make one more comment with the conclusion?

rikkertkoppes commented 11 years ago

Let's discuss the remaining points Monday.

About internationalization and error messages: I think we should all strings from the XML and replace them by keys. We will end up with one (or two) language file which will be used for translating both the XML and the rest of the interface. There are standards in XML and json to do this. Filling it in would be a very simple Xsl transformation

About the water items constraints: this is indeed a tough one if we want to cover it completely. Especially in XML. Let's discuss monday.

@kmeesters we need mission definition discussion time on Monday.

kmeesters commented 11 years ago

I'll put it on the agenda. @robvw will you be present and/o will join us for diner?

On the topic, I just wanted to add that the mission seem to developing in this direction, increasing choices for the teams to make. However the question is whether or not we should capture all these constraints in the file. Most of the constraints are implied in the rules or the design of the missions. For example there are only 12 items on the mat, so it is not possible there will ever be more then 12 in one or the other. The only thing that can go wrong is wrong input by the referee. But that is always possible.

@robvw The penalties are an example of this. There are only 4 penalties debris pieces. that can move to 8 positions, so it will not be possible to have more then 8 penalties. Ergo, not defined in the rules, but designed into the mat.

And for my other remarks, there are quite some points I'd like to suggest to Scott (clarity in general and consideration for non English-speaking in particular), which I think may work better over a beer. If possible at all :)

On the translation: my idea would be not add to add additional language files, but let people just modify the XML.

robvw commented 11 years ago

Yes, I'll be present on Monday, didn't I sign up yet?

One thing I want to discuss tomorrow: should we define defaults (the situation at the start of the match) so referees only have to fill out what changes (less work for low-scoring teams), or should we initialise to an empty sheet so referees are forced to fill out everything for every match (lower risk of missing something for high-scoring teams)?

@kmeesters Actually, guarding against referee error is exactly the reason I want to define those constraints. Referees are human too and they do make mistakes (some years, the scoring sheet uses an unfortunate phrasing and a lot of errors are made).

This particular example is unlikely to go wrong (well, it's just as likely to go wrong in a way we cannot detect: I'd be suspicious of a form listing a total of eleven objects), but look at last year. There were questions "Robot Only Touches Slanted Platform", "Robot Only Touches Balanced Platform" and "Platform Only Touches Robot and Mat". If the robot is on the slanted platform: 45 points, if the robot is on the balanced platform: 65 points... but only if the platform only touches the robot and the mat: if a strategic object touches it (presumably to keep it balanced), no points are awarded.

There is no benefit for a team to bring a strategic object: failing to balance still awards 45 points, but using the strategic object always results in no points at all. (One, rather obscure corner case exists; long story, I'll bore you with it tomorrow. :p ) But I still got half a dozen score sheets with "Platform Only Touches Robot and Mat" filled out as "No". In every case, going back to the referee caused them to correct it...

Admittedly that is technically a bad example (this situation is unlikely but can indeed happen; constraints can only prevent combinations which are utterly impossible), but it does show referees do make errors.

@kmeesters The score sheet you sent a screenshot of contains a heading ("mission") called "penalty". This contains three questions ("observables"). The first is indeed the penalties for debris pieces. But I was talking about the second and third one, about "small junk" and "large junk". If I understood the challenge description correctly, in particular Rule 12, this refers to "strategic objects": models brought by teams which aid in accomplishing goals (for instance, a ramp, to make crossing an obstacle easier).

I emailed Scott asking what happens when a team uses nine strategic objects (this is an unlikely scenario, but if FLL taught me one thing, it is to expect the unexpected): whether the referee just marks "8" on the scoring form (allowing the ninth penalty "free of charge"), or whether to write down the actual number instead (and give the penalty points nine times). In the last case... we're in trouble (well, we aren't; we can just hack around in the database and fix things, but other tournaments using our software would have a problem).

rikkertkoppes commented 11 years ago

I removed the complete xml from the wiki and put it in the repo: db4fc4899b790607f451fc3e26f4f235842c7ea4

robvw commented 11 years ago

Just got a reply from Scott. He called the sheet Kenny showed us "sort of wrong" and sent me his own tool (an Excel sheet). That one uses a maximum of twenty penalties, but the email sounded like it is actually supposed to be unrestricted (in theory at least, in practice a team getting twenty penalties is already deemed "insanely unlikely").

Question: do we want to have support for an unrestricted field (for instance, if the official sheet would use "write a number" instead of "circle one of these answers"), or does that break too much of the design for a too-unlikely situation? Hack-ish solution would be "it can always be fixed in admin mode".

kmeesters commented 11 years ago

Hmm, a difficult one. A few thought from my side:

First off, FIRST should become really more accurate in what they communicate. It will be hard for developers, partners and teams to understand and process these inaccuracies. Especially for teams whose primary language might not be English. Clear and simple (not simplistic) rules with clear (score) definitions and options (i.e. not things like 'insanely unlikely). Well anyway, end of rant :)
I know the Excel from Scott, and it actually illustrates my previous point, apparently FIRST doesn't know either and communicates different things. Which bring means me to my main point: our point of reference.

I would opt for sticking to what FIRST released. Of course we can see some holes in the products they output, but that means that others will most likely identify those gaps as well. This usually triggers an update in the materials. Which could be a new versions of the score-sheet (but doesn't have to be, it can also be a mission update). It would suggest sticking to the official current docs. (in this case the score-sheet).

Depending on how far we in the season an update of the XML for that season could be released, which hopefully then also produce updated printable scores-sheets. For me it is not necessary to have one XML file. Countries, teams and tournaments will adapt it anyway to fit their needs. That is also one of the reasons to use XML and not hard-code the missions. On the other hand wen can still maintain a 'master copy' on our website.
I imagine the admin mode does not allow modification outside of what the XML dictates. I.e. the admin mode allows modification of the observations after submitting them. But I do expect you certainly cannot modify the points directly. The only option would be to change the missions definitions in the XML to allow another observable situation (which we should prohibit as well). However I may be wrong here.

Hopefully I time, we can convince FIRST to use the XML scheme when designing their mission :). Perhaps we can talk a bit with Scott about that at some point. Ideally they would sign off on one the XML files.

On a side note: the Excel he built illustrates the need for a single device administration tool.

robvw commented 11 years ago

Actually, the info to the teams ("13 penalty points for each junk item you leave on the table; no maximum specified") is clear and seems to be what is intended. Note the phrase "insanely unlikely" comes from a (private) email, not the (public, official) Q&A on the website.

The exact power of the admin interface is a design decision. I would actually like it to be able to overrule a score. Sure, it's an option you won't use often, but when you do need it, you really, really need it. I'm very much in favour of putting it behind a big confirmation (not "Are you sure? [Yes / No]" but something like "You shouldn't do this!! [Cancel / This is an emergency; I really know what I'm doing]"), but I do think it should be possible. (It is already possible: the format Rikkert uses for information exchange and storage has an additional "total score" field. You could close the scoring application, edit the file and restart the system, at which point the modified file is read in again, presumably without re-calculating and overwriting the "total score" field.)

It'll probably be a while before FIRST officially approves one of our XMLs, let alone restrict themselves to what is easily expressible in that format. It seems more realistic (and more useful) to convince them to follow the general ideas behind it: explicitly list all possible situations and the corresponding scores. This not only help us, but also everyone else writing scoring software and probably even the teams. For most missions it won't matter too much, but for the penalties, including the following table would have solved the issue right away by explicitly covering the corner case:

Big junk items     Score
0                     0
1                   -13
2                   -26
3                   -39
4                   -52
5                   -65
6                   -78
7                   -91
8 or more          -104

btw, such a table would also have prevented the issue with the unclear scoring of the touch penalties. This design principal would certainly help us, but we don't have to ask FIRST (Scott) to adopt it because of our system; it would be perfectly explainable as a limitation of (paper) scoring sheets.

Apologies if this is getting off-topic; would it be better to discuss this in another ticket (or not on github at all)?

kmeesters commented 11 years ago

Hi,

Yes, I think it is indeed a wise idea to keep the discussion off Github (for now). How ever it is interesting debate to have, getting FIRST informed about our efforts is wise, we can discuss how they and other can benefit the most. But let's indeed save it for a face-to-face meeting.

Furthermore, I do appreciate the table, a recurring element int he FLL (although not explicit this year).

And, regarding the edit of scores, if it is possible to neatly implement a manual score entry, (with indeed proper warnings) I agree to implement this feature in admin mode. However it should be only * after * one cannot make the changes by editing the form, but we maybe we are talking now only about the 'extreme' scenario.

FirstLegoLeague / fllscoring

Progress on field definitions #4