openworm / tracker-commons

Compilation of information and code bases related to open-source trackers for C. elegans
11 stars 12 forks source link

Time-series plate features? #19

Closed MichaelCurrie closed 8 years ago

MichaelCurrie commented 8 years ago

The below section of the WCON specification discusses "plate features".

However, this does not appear to allow for time-series plate features, such as total worm area. (potentially across multiple worms), or (I'm making this one up) plate temperature (over time).

I suggest we allow for root['data'] elements that have no "id":... entry. Those entries will be considered to be adding data for the whole plate at the specified time.

What do you think?

For features that describe an entire plate rather than properties of individual worms (such as density or aggregate number), the features object can be added outside of data:

{
    "tracker-commons":true,
    "units":{"t":"s", "x":"mm", "y":"mm"},
    "data":[
        "@OMG":{
               "feature_names":["speed", "curvature", "width"],
               "feature_units":["mm/s", "1/mm", "mm"]
        },
        { "id":1,
          "t":1.3,
          "x":[1215.11, ...],
          "y":[234.89, ...],
          "@OMG":{ "feature_values":[0.34, 1.5, 0.873] }
        }
        ...
    ]
    "@OMG":{
          "plate_feature_names":["density", "aggregate number"],
          "plate_feature_units":["1/mm^2", null],
          "plate_feature_values":[0.1, 8]
    }
}
Ichoran commented 8 years ago

If someone wants these global features, they can add it in an @-section at the same level as data according to the current specification. The specification also doesn't promise that a data section corresponds to a single plate, though that is a natural organization. For instance, in a system tracking 6 plates with a single camera, there might be an additional "@FOO": { "plate": [1, 2] } entry on each separate animal.

I think the idea was that these sorts of derived features would be derivable from the underlying data by various different people (possibly with various different results due to differences in analysis method), and therefore should go under lab-specific codes.

However, there is no particular reason why the Tracker Commons implementations can't offer support for some of those codes. They're just not guaranteed. (So, for instance, I won't put any of them into the Scala reference implementation, though I will probably put some into yet another implementation which is for high-performance parsing and writing. Whether that goes into this repository or is part of Choreography, I've not yet decided.)

Anyway, the syntax does have to be valid JSON, so [ "@FOO": "whatever", { "id" = 1 } ] is right out. Arrays are bare values. I guess it's supposed to be [ { "@FOO": "whatever" }, { "id" = 1 } ]?

MichaelCurrie commented 8 years ago

For instance, in a system tracking 6 plates with a single camera, there might be an additional "@FOO": { "plate": [1, 2] } entry on each separate animal.

That's a really good point, that I hadn't considered. For now in my implementation I'll first ignore anything in data that isn't t, x, y, ox, oy, head, ventral, or id.

Anyway, the syntax does have to be valid JSON, so [ "@FOO": "whatever", { "id" = 1 } ] is right out. Arrays are bare values. I guess it's supposed to be [ { "@FOO": "whatever" }, { "id" = 1 } ]?

Yes I agree. So there are a couple of typos in the spec that I'm just going to take the liberty of correcting via git push right now: