search

VlSomers / keypoint_promptable_reidentification

[ECCV24] Keypoint Promptable Re-Identification: SOTA ReID method robust to occlusions and multi-person ambiguity
Other
87 stars 5 forks source link

Hi #1

Open pxczzz opened 3 months ago

pxczzz commented 3 months ago

Hi, @VlSomers, could you please help me understand why my process of extracting features from the gallery set gets stuck around 85% during testing? I think it might have been killed by the system.

VlSomers commented 3 months ago

Hi @pxczzz, all extracted features are stored in your RAM during evaluation, so you need enough memory to support it. How much memory do you have?

pxczzz commented 3 months ago

Thanks for your response. I used four 16GB NVIDIA Tesla T4 GPUs during the testing process, and the dataset I used is Occluded-Duke.

VlSomers commented 2 months ago

And how much memory do you have? (Not GPU memory, but main memory)

pxczzz commented 2 months ago

It's something like this： Total Memory: 125GB Used Memory: 27GB Free Memory: 21GB Shared Memory: 149MB Buffers/Cache: 76GB Available Memory: 97GB

VlSomers commented 2 months ago

This seems enough, are you sure your process is allocated that much memory?

pxczzz commented 2 months ago

Could you please help me understand why, when my testing process gets stuck, it can consume over 70% of the system's main memory?

VlSomers commented 2 months ago

This is because the inference procedure implemented in Torchreid will first extract all embedding (reid features) for all images in the test set (query + gallery), keep them in the main memory, and then compute the distance matrix between all queries and all galleries. At some point during the inference procedure, you therefore have reid features for 10.000+ images stored in your memory. This is something that should be optimized.