search

poojahira / image-captioning-bottom-up-top-down

PyTorch implementation of Image captioning with Bottom-up, Top-down Attention
164 stars 39 forks source link

Is there any requirement on the memory? How to solve the following issue? Thanks #6

Closed jiangguoding closed 5 years ago

jiangguoding commented 5 years ago

My computer configuration: GTX1080 8G
Memory: 32G

env_pytorch_updown36) world@world:~/code4/poojahira/captioning$ python train.pyEpoch: [0][0/5665] Batch Time 9.657 (9.657) Data Load Time 2.394 (2.394) Loss 269.4812 (269.4812) Top-5 Accuracy 0.000 (0.000) Epoch: [0][100/5665] Batch Time 2.748 (2.616) Data Load Time 2.173 (2.094) Loss 39.0252 (41.8164) Top-5 Accuracy 34.391 (31.579) Epoch: [0][200/5665] Batch Time 1.089 (2.319) Data Load Time 0.673 (1.827) Loss 37.9037 (40.1628) Top-5 Accuracy 35.671 (33.439) Epoch: [0][300/5665] Batch Time 1.286 (1.926) Data Load Time 0.803 (1.442) Loss 39.6853 (39.3100) Top-5 Accuracy 38.070 (34.595) Epoch: [0][400/5665] Batch Time 0.845 (1.712) Data Load Time 0.347 (1.237) Loss 39.3415 (38.8955) Top-5 Accuracy 36.847 (35.391) Traceback (most recent call last): File "train.py", line 306, in main() File "train.py", line 96, in main epoch=epoch) File "train.py", line 172, in train loss.backward() File "/home/world/miniconda3/envs/env_pytorch_updown36/lib/python3.6/site-packages/torch/tensor.py", line 102, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/home/world/miniconda3/envs/env_pytorch_updown36/lib/python3.6/site-packages/torch/autograd/init.py", line 90, in backward allow_unreachable=True) # allow_unreachable flag RuntimeError: CUDA out of memory. Tried to allocate 173.88 MiB (GPU 0; 7.93 GiB total capacity; 6.18 GiB already allocated; 50.31 MiB free; 377.16 MiB cached) (env_pytorch_updown36) world@world:~/code4/poojahira/captioning$ (env_pytorch_updown36) world@world:~/code4/poojahira/captioning$

jiangguoding commented 5 years ago

RuntimeError: CUDA out of memory. Tried to allocate 173.88 MiB (GPU 0; 7.93 GiB total capacity; 6.18 GiB already allocated; 50.31 MiB free; 377.16 MiB cached)

jiangguoding commented 5 years ago

By the way: Could help upload the "BEST_34checkpoint_coco_5_cap_per_img_5_min_word_freq.pth.tar". So that we can "python eval.py" to see the effects first? Thanks.

poojahira commented 5 years ago

Hi, the memory I requested for on my university cluster is 120 GB.

poojahira commented 5 years ago

The pretrained model is available in the Results section of my README.

jiangguoding commented 5 years ago

OK. Thanks for your response indeed.

jiangguoding commented 5 years ago

Thanks.