Closed erotavlas closed 4 years ago
The final error is just a side effect when there's an error in training, the relevant warning is further up:
D:\Anaconda\envs\spacy\lib\site-packages\torch\nn\modules\rnn.py:54: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.2 and num_layers=1
I tried again with a value greater than 1 --bilstm-depth 2
but it still fails
Itn NER Loss NER P NER R NER F Token % CPU WPS GPU WPS
--- --------- ------ ------ ------ ------- ------- -------
0%| | 0/465870 [00:00<?, ?it/s]D:\Anaconda\envs\spacy\lib\site-packages\torch\nn\modules\rnn.py:522: RuntimeWarning: RNN module weights are not part of single contiguous chunk of memory. This means they need to be compacted at every call, possibly greatly increasing memory usage. To compact weights again call flatten_parameters().
self.dropout, self.training, self.bidirectional, self.batch_first)
⚠ Aborting and saving the final best model. Encountered exception: Input and
parameter tensors are not at the same device, found input tensor at cuda:0 and
parameter tensor at cpu
✔ Saved model to output directory
test\model-final
⠸ Creating best model...
Traceback (most recent call last):
File "D:\Anaconda\envs\spacy\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "D:\Anaconda\envs\spacy\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\Anaconda\envs\spacy\lib\site-packages\spacy\__main__.py", line 33, in <module>
plac.call(commands[command], sys.argv[1:])
File "D:\Anaconda\envs\spacy\lib\site-packages\plac_core.py", line 367, in call
cmd, result = parser.consume(arglist)
File "D:\Anaconda\envs\spacy\lib\site-packages\plac_core.py", line 232, in consume
return cmd, self.func(*(args + varargs + extraopts), **kwargs)
File "D:\Anaconda\envs\spacy\lib\site-packages\spacy\cli\train.py", line 586, in train
best_model_path = _collate_best_model(final_meta, output_path, best_pipes)
File "D:\Anaconda\envs\spacy\lib\site-packages\spacy\cli\train.py", line 652, in _collate_best_model
path2str(best_component_src / component), path2str(best_dest / component)
TypeError: unsupported operand type(s) for /: 'NoneType' and 'str'
Hi @erotavlas : It looks like there's some CPU/GPU incompatibility. This is difficult to debug with just the single message. Could you go to the script spacy.cli.train.py
and around line 566, where it catches the Exception
and prints the log, instead of the msg.warn("Aborting...)
bit, could you type raise e
, run your command again, and show the full error stack trace here?
This issue has been automatically closed because there has been no response to a request for more information from the original author. With only the information that is currently in the issue, there's not enough information to take action. If you're the original author, feel free to reopen the issue if you have or find the answers needed to investigate further.
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
spacy version 2.2.4
when added option
--bilstm-depth 1
to the end of train command in CLI I get the following error