search

jiesutd / LatticeLSTM

Chinese NER using Lattice LSTM. Code for ACL 2018 paper.
1.79k stars 457 forks source link

Out of Memory问题 #68

Closed AaronWhite95 closed 5 years ago

AaronWhite95 commented 5 years ago

运行main.py后出现如下问题 build batched crf... THCudaCheck FAIL file=/pytorch/torch/lib/THC/generic/THCStorage.cu line=58 error=2 : out of memory Traceback (most recent call last): File "main.py", line 458, in train(data, save_model_dir, seg) File "main.py", line 252, in train model = SeqModel(data) File "/home/xfbai/LatticeLSTM/model/bilstmcrf.py", line 24, in init self.crf = CRF(label_size, self.gpu) File "/home/xfbai/LatticeLSTM/model/crf.py", line 47, in init init_transitions = init_transitions.cuda() File "/home/xfbai/anaconda3/envs/py2/lib/python2.7/site-packages/torch/_utils.py", line 69, in _cuda return newtype(self.size()).copy(self, async) RuntimeError: cuda runtime error (2) : out of memory at /pytorch/torch/lib/THC/generic/THCStorage.cu:58 请问是什么问题呢，试过网上很多方法都没用，batch_size已经是1了。 谢谢。

jiesutd commented 5 years ago

显示的是out of memory 啊，是不是你内存太小了或者数据太大了？用小的sample 数据实验看看

AaronWhite95 commented 5 years ago

小的数据可以，但是要训练大数据怎么办呢，batch_size一直是1

jiesutd commented 5 years ago

那得看你数据有多大了。如果大到 out of memory， 那就得自己改代码优化内存占用了。