search

JeffersonLab / coatjava

1 stars 20 forks source link

feat: merge a set of HIPO files to a smaller set #377

Closed c-dilks closed 2 days ago

c-dilks commented 5 days ago

This tool merges a set of input HIPO files to a set of output HIPO files, where you may control the number of input files per output file; for example, use this tool if you have 1000 small HIPO files but would rather have 10 large HIPO files.

gavalian commented 5 days ago

Hipo utils can split the file into smaller chunks, so one could merge a file and then split it into smaller chunks.

c-dilks commented 5 days ago

Hipo utils can split the file into smaller chunks, so one could merge a file and then split it into smaller chunks.

Yes, but this PR's script is more efficient since it just needs to run the merge commands; it can do so either sequentially locally or in parallel with Slurm.