estimate for segmentation.py memory usage

mr-c commented 4 years ago

In the CWL tool description for segmentation.py we need memory sizing hints in mebibytes (2**20).

We can provide this number using a bit of JavaScript.

The following numbers are available for the formula:

number of inputs JSONs (inputs.bins.length) and the size of each (inputs.bins[N].size)
Number of cells per file from --cells-per-file (inputs.cells_per_file)
the size of the --fasta file (inputs.pangenome_sequence.size)
and the number of CPU cores passed to --parallel-cores (runtime.cores, if that matters)

mr-c commented 4 years ago

Any progress on this @dimatr or others?

dimatr commented 4 years ago

still waiting the deNBI cloud to come up - will test big inputs there to get an impression

mr-c commented 4 years ago

Okay, didn't know that was a blocker. You can also run tests on https://biohackathon.curii.com :-)

graph-genome / component_segmentation