Open ctb opened 3 years ago
Oh, and also, we're inconsistent with md5sum output per https://github.com/dib-lab/sourmash/pull/1346#pullrequestreview-610882484
we could/should also consider including the metric used - jaccard, containment, max containment.
and/or just, like, calculate all of those.
(sigh, for scaled sketches; more than jaccard not possible with regular MinHash)
CSVs are hard (impossible?) to version, but we should have some way of doing that too. Or do we just keep ever growing the CSV and never removing columns? :upside_down_face:
thoughts on approach in https://github.com/sourmash-bio/sourmash/issues/1555?
Basically, I think it's OK to pin column names to sourmash versions, with appropriate deprecation approaches and command-line upgrade flags. That fits with their use in workflows.
In manifests, we are using:
# SOURMASH-MANIFEST-VERSION: 1.0
but I'm pretty confident that this breaks pandas/Python header detection, sigh. IMO it was OK to do this for manifests because these are not intended to be end-user-consumable.
https://github.com/sourmash-bio/sourmash/issues/416 has the idea of building standard pandas/CSV loading functions for sourmash output, which is something I'm trying out over in genome-grist for gather output - https://github.com/dib-lab/genome-grist/pull/176. But I'd be loathe to break all CSV readers everywhere :(.
I guess... we could include a "version for this CSV format" in the first column in the first row, and leave that column blank, or something? or do the same but for the last column in the first row (so, less visible, but leaving it blank is less annoying for manual inspection of the CSV). This would make it a header but that's ok.
(Some of this might be 5.0 material, because they change the file format in backwards-incompatible ways)
A few issues --
--max-containment
or--containment
are specifiedrelated to https://github.com/dib-lab/sourmash/issues/1247, https://github.com/dib-lab/sourmash/issues/410, and #448.
It's not really clear what to do here. The addition of prefetch https://github.com/dib-lab/sourmash/pull/1370 might provide a useful alternative here, and/or we could provide JSON output that has more ...flexibility per https://github.com/dib-lab/sourmash/issues/448.