when I run section #overfit a sample# in train_model.ipynb, it appears. How can I solve it?

Viredery / tf-eager-fasterrcnn

Faster R-CNN R-101-FPN model was implemented with TensorFlow2.0 eager execution.

MIT License

89 stars 46 forks source link

when I run section #overfit a sample# in train_model.ipynb, it appears. How can I solve it? #13

Open ily666666 opened 4 years ago

ily666666 commented 4 years ago

when I run section #overfit a sample# in train_model.ipynb, it appears. How can I solve it? OOM when allocating tensor with shape[1,1024,64,64] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [Op:FusedBatchNormV3]

Viredery commented 4 years ago

It looks like your GPU memory is not sufficient to run this network. Maybe change backbone to ResNet50 or ResNet18. However I don't have relative pre-trained weights unfortunately.