openworm / tracker-commons

Compilation of information and code bases related to open-source trackers for C. elegans
11 stars 12 forks source link

Binary data has no obvious way to be embedded or associated with WCON files #159

Open Ichoran opened 7 years ago

Ichoran commented 7 years ago

We should recommend a way to deal with binary data that may or may not be appropriate to embed in WCON so that different readers have nonzero hope of finding and interpreting it, and especially can carry along associated data if these files exist (or serve as a warning to someone examining the files manually that associated data exists).

One idea is to have an external or extra tag in files that would list associated files. Alternatively, this information could be placed in metadata. For embedded binary data, recommending Base64 is probably a good idea, either with official support or as an example.

MichaelCurrie commented 7 years ago

Discussion today with @aexbrown @Ichoran favoured the external tag approach rather than including binary data within the WCON file proper to keep the WCON file somewhat human-readable. I proposed calling it manifest but perhaps that's too grandiose.

Ichoran commented 7 years ago

If we just want a way to point at external files, it can go in the files object, as something other than current, prev, or next. I think maybe we actually want two such things, one which is a list of files that is expected to be found with the wcon files like prev and next are expected to be, and another which is a general set of links to whatever. We could call these extra and references respectively. So

"files": {
  "current":"Worm data 2.wcon",
  "next":["Worm data 3.wcon"],
  "prev":["Worm data 1.wcon"],
  "extra":["Worm data initial picture.tiff"],
  "references":["http://github.com/opemworm/tracker-commons"]
}

The idea would be that stuff in extra ought to come along in zip files, but stuff in references should just be some arbitrary meaningful pointer to something else.