search

roytseng-tw / Detectron.pytorch

A pytorch implementation of Detectron. Both training from scratch and inferring directly from pretrained Detectron weights are available.
MIT License
2.82k stars 565 forks source link

Memory grows with less TEST.SCALE #144

Open JudyYe opened 6 years ago

JudyYe commented 6 years ago

Hi,

I observed a weird memory growth when I set the TEST.SCALE down from 800 to 600 or 200, consuming memory 872MiB(800), 1504MiB(600), 3035Mi(200).

The default config is configs/baselines/e2e_faster_rcnn_R-50-C4_1x.yaml with python3.6, Telsa K40, single GPU.

It happens to me in both training and inference time. You might want to reproduce as :

python tools/infer_simple.py \
     --dataset coco \
     --cfg configs/baselines/e2e_faster_rcnn_R-50-C4_1x.yaml \
     --load_detectron data/pretrained_model/faster_r50_coco.pkl \
     --image_dir any_image_dir/  \
     --output_dir $output \
     --set TEST.SCALE $SCALE

Does anybody else have a similar result? Any clue why this happens?

Thank you.

SpyderXu commented 5 years ago

The GPU memory consuming is strange for me too.