search

lely475 / ocelot23algo

Approach that won 3rd place in the OCELOT 2023 Challenge. Multi-organ H&E-based deep learning model for cell detection, applicable for tumor cellularity/ purity/ content estimation.
https://arxiv.org/abs/2312.12151
Other
7 stars 0 forks source link

MemoryError #1

Closed FengZhongLiuDong closed 4 months ago

FengZhongLiuDong commented 4 months ago

I apologize for any confusion earlier. Here is the translation of your message into English:

"May I ask what your computer memory configuration is? My computer has 32GB of memory, but I am encountering memory errors when running the program.Is it because the WSI is too large? Also, regarding the code line 'data_path = "" # Add data directory containing (only!) wsis', does the WSI in the data directory need to be complete slices?"

I hope this translation is helpful. Please let me know if you have any further questions or if there is anything else I can assist you with!

lely475 commented 4 months ago

Hi, And thank you for reaching out.

1. RAM requirements

"May I ask what your computer memory configuration is? My computer has 32GB of memory, but I am encountering memory errors when running the program.Is it because the WSI is too large?

I have previously run the code on 2 machines with the following specifications: Hardware Type Machine 1 Machine 2
RAM 128 GB 128 GB
GPU NVIDIA Titan V (12GB memory) NVIDIA GeForce RTX 3090 (24GB memory)
CPUs 32 cores (Intel Xeon Gold 6142) 20 cores (Intel i9-10900X)

So I think your assumption was correct that the WSI was too large to fit into memory. I was currently loading the full WSI into memory before tiling it and running inference on a batch of tiles. I have now adapted the code to only load the current batch of tiles during inference, this should reduce memory consumption significantly.

Please let me know if this change solves your problem!

2. WSI format

'data_path` = "" # Add data directory containing (only!) wsis', does the WSI in the data directory need to be complete slices?"

Can you clarify what is meant by "complete slices"? Currently any WSI format that you can open with openslide is supported (list of supported fileformats by openslide).

FengZhongLiuDong commented 4 months ago

Thank you for your assistance. Your detailed explanation has resolved my confusion. By "complete slices" I mean whether it refers to a complete whole-slide image or images that have been divided into smaller patches (1024*1024)?

lely475 commented 4 months ago

Glad to hear that, and I see, the algorithm expects the whole-slide image as input.