search

AstarLight / Lets_OCR

A repository for OCR, which inlcudes some classical OCR algorithms Pytorch implementation such as CTPN, EAST and CRNN.
MIT License
653 stars 327 forks source link

关于运行内存的问题 #50

Open harrymore opened 5 years ago

harrymore commented 5 years ago

我在训练模型的时候，使用gpu进行训练，但是还没开始输出loss数据，训练机的内存(16G)就全部给吃掉了，最后报“OSError: [Errno 12] Cannot allocate memory”的错误，请问训练这个模型的时候需要用多大的内存呢？

harrymore commented 5 years ago

报错信息： image scale: [32,160] model_save_path: ./w160_bs64_model gpu_id: 1 batch_size: 64 Using cuda alphabet class num is 5990 CRNN( (cnn): Sequential( (vgg_16): Vgg_16( (convolution1): Conv2d(1, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, (pooling1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mo (convolution2): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1 (pooling2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mo (convolution3): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=( (convolution4): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=( (pooling3): MaxPool2d(kernel_size=(1, 2), stride=(2, 1), padding=0, dilation= (convolution5): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=( (BatchNorm1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_ru (convolution6): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=( (BatchNorm2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_ru (pooling4): MaxPool2d(kernel_size=(1, 2), stride=(2, 1), padding=0, dilation= (convolution7): Conv2d(512, 512, kernel_size=(2, 2), stride=(1, 1)) ) ) (rnn): Sequential( (rnn): RNN( (Bidirectional_LSTM1): LSTM(512, 256, bidirectional=True) (embedding1): Linear(in_features=512, out_features=512, bias=True) (Bidirectional_LSTM2): LSTM(512, 256, bidirectional=True) (embedding2): Linear(in_features=512, out_features=5990, bias=True) ) ) ) /home/kinth/Envs/ocr_crnn/lib/python3.6/site-packages/torch/cuda/init.py:114: U Found GPU0 GeForce RTX 2080 which requires CUDA_VERSION >= 9000 for optimal performance and fast startup time, but your PyTorch was compiled with CUDA_VERSION 8000. Please install the correct PyTorch binary using instructions from http://pytorch.org

warnings.warn(incorrect_binary_warn % (d, name, 9000, CUDA_VERSION)) Traceback (most recent call last): File "train.py", line 156, in train_iter = iter(train_loader) File "/home/kinth/Envs/ocr_crnn/lib/python3.6/site-packages/torch/utils/data/data return _DataLoaderIter(self) File "/home/kinth/Envs/ocr_crnn/lib/python3.6/site-packages/torch/utils/data/data w.start() File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 66, in _launch self.pid = os.fork() OSError: [Errno 12] Cannot allocate memory

不知道是不是pytorch版本问题

GanWang commented 5 years ago

我训练的时候没有遇到问题但是测试的时候CPU会瞬间爆掉，卡住几秒后恢复，每张图片都是的。