search

openworm / tracker-commons

Compilation of information and code bases related to open-source trackers for C. elegans
11 stars 12 forks source link

Initial WCON conversion of Sibernetic output file #149

Closed pgleeson closed 6 years ago

pgleeson commented 7 years ago

As mentioned, this is the txt output from @a-palyanov from Sibernetic along with some Python scripts for plotting and converting to WCON.

It would be great if you could have a look at this @MichaelCurrie, @cheelee and update the WCON output as required. Then this can be added to the Sibernetic repo, and I'll also add an option to automatically make this transformation in sibernetic_c302.py

MichaelCurrie commented 7 years ago

Wait, @pgleeson, you already wrote the script? Nice! @Ichoran, I suspect you'll want this code to be in a different repository to keep tracker-commons only for code that is for reading and writing generic WCON.

So the ultimate home for your script would be in

https://github.com/openworm/open-worm-analysis-toolbox/tree/master/tools/

or as you mention, perhaps in the Sibernetic repo.

I will try to run this.

Ichoran commented 7 years ago

It might not be bad to have a stock set of converters somewhere in the Tracker Commons project. I wouldn't put it directly in the existing language-specific files, but it can certainly live in the project as long as it's not too esoteric.

Ichoran commented 7 years ago

@pgleeson - Maybe you can place this into converters/sibernetic and write in the README.md for the directory an explanation of what it does, which seem to be take files that look like (time) X: (x-value) (x-value) ... (x-value) Y: (y-value) (y-value) ... (y-value) Other-Junk and convert them into a WCON file with a single animal with ID 1. That seems like a useful enough capability (e.g. a good way to get into WCON from someone with an Excel file, for instance!) to be in a general converter, if we are to have general converters here.

pgleeson commented 7 years ago

The stable version of the python script can eventually end up in the Sibernetic repo, but for now it's good if it's in one of yours until you're happy with the output. It might be good to keep an example of WCON generated from sibernetic as one of the standard examples that language specifc bindings are tested against, hence putting it in tests/sibernetic.

Clearly it isn't using the Python API to write WCON, but the JSON was so trivial to write it seemed overkill.. Also it would be one more dependency for someone to install if they wanted the full Sibernetic/python toolchain to work...

pgleeson commented 7 years ago

@Ichoran Certainly the images etc. don't need to end up in openworm/tracker-commons, but might be useful to stay in my fork for the time being until it's stable, then the python script can be sent to Sibernetic, a short WCON can be added to tracker-commons etc.

So don't consider this a PR to be merged for now, just a useful place to work on the sibernetic->wcon script until it's stable. If someone wants to change the code into a generic converter, then go ahead (I'm not convinced of the need...), but there is a more urgent requirement for a working script for handling the Sibernetic conversion.

Ichoran commented 7 years ago

@pgleeson - I'm wondering if you would consider turning it into a general converter :)

The reason is that the data is in an especially simple format, and it may be particularly easy for other people to get data into that format. Then, having a trivial converter to turn the columns of numbers into JSON would be all that's required to get off the ground. Since you've done 80% of the work already, maybe you're willing to do a little more.

(I agree that Sibernetic should be the ultimate home of data it produces and scripts to manipulate that data, possibly including conversion if a simple generic converter can't do enough.)

slarson commented 7 years ago

Pinging this PR -- we could really use this now to begin running analysis on interesting Sibernetic behaviors like this: curvature_plot_2_frequences

Should we maybe merge this and begin iterating and improving the script as a jumping off point?

slarson commented 6 years ago

@cheelee is going to take a crack at this.

pgleeson commented 6 years ago

Best to close this as all the sibernetic->WCON work has been merged into sibernetic codebase here: https://github.com/openworm/sibernetic/blob/development/wcon/generate_wcon.py.

As mentioned, this could certainly be the basis for a general purpose text file to WCON converter, but the specific format sib. uses is dealt with for now (though pending testing with other WCON tools...)