huggingface / evaluate

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
https://huggingface.co/docs/evaluate
Apache License 2.0
1.9k stars 235 forks source link

evaluate consuming Memory and slow down the process #559

Closed Redix8 closed 4 months ago

Redix8 commented 4 months ago

https://huggingface.co/docs/transformers/main/en/tasks/semantic_segmentation#custom-dataset

this page's example(and any other examples online computing miou with hugging face) has problem of slowing down.

it starts with normal speed and after few hundreds steps, starting to take up the memory and slow down

i'm not sure if that's leakage or some kind of bug, but it's not a normal behavior.

it's not a hardware shortage. i monitored it. not a CPU or GPU calculation bottleneck or lack of memory.