Real world WCON example

[JimHokanson]

Do we have any real data in the WCON format? If not, can we get one? I'm not sure how we want to incorporate this code but perhaps we would include a separate folder of these functions. I'd also prefer most of this generic code be in Python ...

More specifically, I'd like a function that reads in some MRC data and outputs a WCON file.

[Ichoran]

Any real data set will be not-small. I can convert one of the MWT data sets, but no experiment worth doing for me will convert to less than 100 MB of WCON. I can always extract a few animals for a minute or so, though.

[JimHokanson]

@Ichoran

I think a large file would really help me to gauge performance. I think this is something that will need to be examined carefully in Matlab to make sure that I have something that is useable. For movement analysis we've uploaded some larger files to a shared Google Drive folder. I think it would be great if you could share one of the larger files that way.

@MichaelCurrie

Can you help with access permissions for OpenWorm Public or do you have an alternative approach in mind? Note, I'm hesitant to share the entire folder just because of the large syncing requirement it imposes on Rex. Although perhaps this is a conversation that we need to discuss in more detail elsewhere ...

Thanks

[MichaelCurrie]

OK I've granted @Ichoran edit access to the Movement Analysis Google Drive folder.

[MichaelCurrie]

Also, I will work on a function of BasicWorm that will save a WCON file. That will accomplish what you are looking for here, @JimHokanson.

[Ichoran]

I have a WCON generator in the Scala implementation's tests that could be used to generate arbitrarily large WCON files with minor modifications at most. I'm also writing a Choreography plugin right now that will let me spit out the data as a WCON file. I'll see how big an example file ends up being. (With just the minimal data, not anything extra like outlines.)

[Ichoran]

I wrote a exporting plugin and it seems to work. (The file can be read by the Scala WCON reader.) The file is called "XJ30_NaCl500mM4uL6h_10m45x10s40s_Ea.wcon". It's in "example_data/MWT example data". For reference, that ~73 MB file takes about 3 seconds to load in the Scala implementation with a "reasonable" machine. (More like 8 on a MacBook Air.)

[JimHokanson]

@Ichoran Great, thanks!

[Ichoran]

@JimHokanson - FWIW, the Julia implementation had bugs in it (I didn't write enough unit tests--this always gets me with dynamically typed and/or non-compiled lanuages!). But I fixed them, and the elapsed time is about 60% longer than Scala (so still pretty good).

[JimHokanson]

@Ichoran @MichaelCurrie Some of the Rex's example contains [0] arrays. Specifically I think the first example is for the 8th index of data (id=12?). Rex, was this intended behavior rather than something like [null null null ... null]? Alternatively, since I think Michael updated the file, I wasn't sure if that was an artifact of the file being updated.

[Ichoran]

@JimHokanson - [0] would mean, in an x or y coordinate, that the animal was only described with one point, and that point is presumably the centroid. Since the number of spine points is not required to be fixed, this seems a reasonable way to indicate a lack of shape data. (Certainly more compact than [null, null, null, ... ].) The MWT does sometimes fail to extract shape data (e.g. if it realized on the second frame that it hadn't caught the entire animal in the region of analysis in the first frame), so that's probably what's going on here.