explosion / spaCy

💫 Industrial-strength Natural Language Processing (NLP) in Python
https://spacy.io
MIT License
29.69k stars 4.36k forks source link

TypeError: unsupported operand type(s) for /: 'NoneType' and 'str' #5683

Closed erotavlas closed 4 years ago

erotavlas commented 4 years ago

spacy version 2.2.4

when added option --bilstm-depth 1 to the end of train command in CLI I get the following error


(spacy) G:\train14>python -m spacy train en dropout_0.5_window_2_lstmdepth_1 finaltrain.json finaltest.json --base-model "1.0.0.2" --pipeline ner --vectors "spacy_word2vecmodel_1.0.0.7"  --n-iter 25 --use-gpu 0 --cnn-window 2 --bilstm-depth 1
Training pipeline: ['ner']
Using GPU: 0
Starting with base model '1.0.0.2'
Adding component to base model 'ner'
Loading vector from model 'spacy_word2vecmodel_1.0.0.7'
Counting training words (limit=0)
D:\Anaconda\envs\spacy\lib\site-packages\torch\nn\modules\rnn.py:54: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.2 and num_layers=1
  "num_layers={}".format(dropout, num_layers))

Itn  NER Loss   NER P   NER R   NER F   Token %  CPU WPS  GPU WPS
---  ---------  ------  ------  ------  -------  -------  -------
  0%|                                                                                       | 0/465870 [00:00<?, ?it/s]D:\Anaconda\envs\spacy\lib\site-packages\torch\nn\modules\rnn.py:522: RuntimeWarning: RNN module weights are not part of single contiguous chunk of memory. This means they need to be compacted at every call, possibly greatly increasing memory usage. To compact weights again call flatten_parameters().
  self.dropout, self.training, self.bidirectional, self.batch_first)
⚠ Aborting and saving the final best model. Encountered exception: Input and
parameter tensors are not at the same device, found input tensor at cuda:0 and
parameter tensor at cpu
✔ Saved model to output directory
dropout_0.5_window_2_lstmdepth_1\model-final
⠼ Creating best model...
Traceback (most recent call last):
  File "D:\Anaconda\envs\spacy\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "D:\Anaconda\envs\spacy\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\Anaconda\envs\spacy\lib\site-packages\spacy\__main__.py", line 33, in <module>
    plac.call(commands[command], sys.argv[1:])
  File "D:\Anaconda\envs\spacy\lib\site-packages\plac_core.py", line 367, in call
    cmd, result = parser.consume(arglist)
  File "D:\Anaconda\envs\spacy\lib\site-packages\plac_core.py", line 232, in consume
    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
  File "D:\Anaconda\envs\spacy\lib\site-packages\spacy\cli\train.py", line 586, in train
    best_model_path = _collate_best_model(final_meta, output_path, best_pipes)
  File "D:\Anaconda\envs\spacy\lib\site-packages\spacy\cli\train.py", line 652, in _collate_best_model
    path2str(best_component_src / component), path2str(best_dest / component)
TypeError: unsupported operand type(s) for /: 'NoneType' and 'str'
adrianeboyd commented 4 years ago

The final error is just a side effect when there's an error in training, the relevant warning is further up:

D:\Anaconda\envs\spacy\lib\site-packages\torch\nn\modules\rnn.py:54: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.2 and num_layers=1
erotavlas commented 4 years ago

I tried again with a value greater than 1 --bilstm-depth 2 but it still fails

Itn  NER Loss   NER P   NER R   NER F   Token %  CPU WPS  GPU WPS
---  ---------  ------  ------  ------  -------  -------  -------
  0%|                                                                                       | 0/465870 [00:00<?, ?it/s]D:\Anaconda\envs\spacy\lib\site-packages\torch\nn\modules\rnn.py:522: RuntimeWarning: RNN module weights are not part of single contiguous chunk of memory. This means they need to be compacted at every call, possibly greatly increasing memory usage. To compact weights again call flatten_parameters().
  self.dropout, self.training, self.bidirectional, self.batch_first)
⚠ Aborting and saving the final best model. Encountered exception: Input and
parameter tensors are not at the same device, found input tensor at cuda:0 and
parameter tensor at cpu
✔ Saved model to output directory
test\model-final
⠸ Creating best model...
Traceback (most recent call last):
  File "D:\Anaconda\envs\spacy\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "D:\Anaconda\envs\spacy\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\Anaconda\envs\spacy\lib\site-packages\spacy\__main__.py", line 33, in <module>
    plac.call(commands[command], sys.argv[1:])
  File "D:\Anaconda\envs\spacy\lib\site-packages\plac_core.py", line 367, in call
    cmd, result = parser.consume(arglist)
  File "D:\Anaconda\envs\spacy\lib\site-packages\plac_core.py", line 232, in consume
    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
  File "D:\Anaconda\envs\spacy\lib\site-packages\spacy\cli\train.py", line 586, in train
    best_model_path = _collate_best_model(final_meta, output_path, best_pipes)
  File "D:\Anaconda\envs\spacy\lib\site-packages\spacy\cli\train.py", line 652, in _collate_best_model
    path2str(best_component_src / component), path2str(best_dest / component)
TypeError: unsupported operand type(s) for /: 'NoneType' and 'str'
svlandeg commented 4 years ago

Hi @erotavlas : It looks like there's some CPU/GPU incompatibility. This is difficult to debug with just the single message. Could you go to the script spacy.cli.train.py and around line 566, where it catches the Exception and prints the log, instead of the msg.warn("Aborting...) bit, could you type raise e, run your command again, and show the full error stack trace here?

no-response[bot] commented 4 years ago

This issue has been automatically closed because there has been no response to a request for more information from the original author. With only the information that is currently in the issue, there's not enough information to take action. If you're the original author, feel free to reopen the issue if you have or find the answers needed to investigate further.

github-actions[bot] commented 2 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.