Option to avoid reloading weights at every inference

wasserth / TotalSegmentator

Tool for robust segmentation of >100 important anatomical structures in CT and MR images

Apache License 2.0

1.55k stars 256 forks source link

Option to avoid reloading weights at every inference #383

Open Kirscher opened 3 weeks ago

Kirscher commented 3 weeks ago

Hello,

I am using TotalSegmentator for multiple inferences and noticed that the weights are reloaded at each inference. This significantly increases the runtime and memory usage.

Would there be an option to keep the weights in memory between inferences to optimize performance?

Thank you for your attention to this request.

Best regards

wasserth commented 2 weeks ago

Unfortunately supporting this would require a large refactoring of the code base. Therefore, this did not happen so far.

Kirscher commented 2 weeks ago

Hi @wasserth

Thanks for the update. I understand the challenge with refactoring.

Also, on another note, regarding dropout: would it be possible to introduce it easily into the project to obtain a distribution of segmentations? If so, I'd be happy to work on it and propose a PR.

Thanks for your feedback