search

NKI-AI / ahcore

Ahcore is the AI for Oncology core computational pathology toolkit
Apache License 2.0
15 stars 1 forks source link

Can we have a callback that will handle h5 file deletion after they've been used? #18

Open AjeyPaiK opened 11 months ago

AjeyPaiK commented 11 months ago

Is your feature request related to a problem? Please describe. When I use the H5 writer to store the intermediate segmentation outputs, they create enormous files. Check below for the h5 files ahcore creates after a segmentation inference on 11 whole slide images. This is not good. We should delete h5 files once their purpose is served (for ex: delete h5 files once the corresponding tiff images have been written.)

image

jonasteuwen commented 11 months ago

In the tiff writer callback you can add the functionality delete_source_h5_after_completion, and make sure the WsiMetric callback is before the deletion

AjeyPaiK commented 11 months ago

I've addressed this in the PR I raised here

AjeyPaiK commented 10 months ago

After more discussion with @jonasteuwen, this issue can become a better feature. In that, we could think of a DeleteH5Files callback. Therefore, changing the title of the issue.