Closed jrvc closed 4 years ago
ooook.... this is embarrassing. I just realized that I had to detach the variable, so GPU memory could be freed. This does the trick:
encoded_sentences.append( [x.detach().to('cpu') for x in model_outputs[4]+model_outputs[1]] )
sorry for the trouble ;) and thanks for the repo and all your hard work
🐛 Bug
Information
I am extracting the internal representations of some of the Marian models. There seems to be a memory leak problem. In this issue, you will find code for running the model sentence by sentence (bsz = 1) just to keep it simple. When I use batching, the problem persists and arises earlier.
Model I am using: MarianMT
modelnames=[f'Helsinki-NLP/opus-mt-en-{tgt}' for tgt in ['de', 'fr', 'ee', 'sv', 'el', 'fi', 'cs', 'ru' ]]
Language I am using the model on: en-{tgt]
The problem arises when using:
[ ] a mix of official example scripts and my own: on this code, I keep the lines used to see if it is a memory problem. Hence the
empty_cache()
, keeping track of the memory usage withmemory_stats()
, and passing things to 'cpu' (but this has not solved the problem for me)The tasks I am working on is:
To reproduce
Steps to reproduce the behavior:
sentences = [] for sent in samples: sent = sent.strip() sent = re.findall(r'[\w]+|.|,|\?|!|;|:|\'|(|)|/',sent) sentences.append(sent)
RuntimeError('CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 31.75 GiB total capacity; 30.67 GiB already allocated; 17.69 MiB free; 30.67 GiB reserved in total by PyTorch)')
533 535 779 811 1025 1057 1271 1303 1517 1549 1763 1795 2009 2041 2255 2287 2501 2533 2747 2779 2993 3025 ... 9635 9667 9881 9913 10127 10159 10373 10405 10619 10651 ... 921311 921343 921557 921589 921803 921835 922049 922081 922295 922327 922541 922573