[feature] Writing results to file in chunks + allow append to existing - Githubissues

JudeWells / chainsaw

MIT License

27 stars 2 forks source link

[feature] Writing results to file in chunks + allow append to existing #10

Closed sillitoe closed 1 year ago

sillitoe commented 1 year ago

Adds features around writing results to file to improve efficiency when running on HPC

write results in buffered chunks (e.g. flush every 20 results)
allows appending to existing results (will skip previously computed results)
does not keep file handles open (opens/appends/closes on flush)