search

deeppavlov / DeepPavlov

An open source library for deep learning end-to-end dialog systems and chatbots.
https://deeppavlov.ai
Apache License 2.0
6.68k stars 1.15k forks source link

NER problems with download=False #881

Closed bavadim closed 5 years ago

bavadim commented 5 years ago

This code build_model(configs.ner.ner_rus, download=False) leads to following exception:

2019-06-14 14:51:45.158 INFO in 'deeppavlov.models.embedders.fasttext_embedder'['fasttext_embedder'] at line 67: [loading fastText embeddings from `/root/.deeppavlov/downloads/embeddings/lenta_lower_100.bin`]
INFO:deeppavlov.models.embedders.fasttext_embedder:[loading fastText embeddings from `/root/.deeppavlov/downloads/embeddings/lenta_lower_100.bin`]
2019-06-14 14:51:45.177 ERROR in 'deeppavlov.core.common.params'['params'] at line 110: Exception in <class 'deeppavlov.models.embedders.fasttext_embedder.FasttextEmbedder'>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/deeppavlov/core/common/params.py", line 104, in from_params
    component = cls(**dict(config_params, **kwargs))
  File "/usr/local/lib/python3.6/dist-packages/deeppavlov/models/embedders/abstract_embedder.py", line 56, in __init__
    self.load()
  File "/usr/local/lib/python3.6/dist-packages/deeppavlov/models/embedders/fasttext_embedder.py", line 68, in load
    self.model = fastText.load_model(str(self.load_path))
  File "/usr/local/lib/python3.6/dist-packages/fastText/FastText.py", line 303, in load_model
    return _FastText(path)
  File "/usr/local/lib/python3.6/dist-packages/fastText/FastText.py", line 37, in __init__
    self.f.loadModel(model)
ValueError: /root/.deeppavlov/downloads/embeddings/lenta_lower_100.bin cannot be opened for loading!
ERROR:deeppavlov.core.common.params:Exception in <class 'deeppavlov.models.embedders.fasttext_embedder.FasttextEmbedder'>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/deeppavlov/core/common/params.py", line 104, in from_params
    component = cls(**dict(config_params, **kwargs))
  File "/usr/local/lib/python3.6/dist-packages/deeppavlov/models/embedders/abstract_embedder.py", line 56, in __init__
    self.load()
  File "/usr/local/lib/python3.6/dist-packages/deeppavlov/models/embedders/fasttext_embedder.py", line 68, in load
    self.model = fastText.load_model(str(self.load_path))
  File "/usr/local/lib/python3.6/dist-packages/fastText/FastText.py", line 303, in load_model
    return _FastText(path)
  File "/usr/local/lib/python3.6/dist-packages/fastText/FastText.py", line 37, in __init__
    self.f.loadModel(model)
ValueError: /root/.deeppavlov/downloads/embeddings/lenta_lower_100.bin cannot be opened for loading!
Traceback (most recent call last):
  File "/opt/aidoc/src/server.py", line 141, in <module>
    load_nodes()
  File "/opt/aidoc/src/server.py", line 137, in load_nodes
    nodes[node_name] = NodeModels(os.path.join(root_data_path, node_name), ft_model, morph)
  File "/opt/aidoc/src/server.py", line 103, in __init__
    self.NER = build_model(configs.ner.ner_rus, download=False)
  File "/usr/local/lib/python3.6/dist-packages/deeppavlov/core/commands/infer.py", line 61, in build_model
    component = from_params(component_config, mode=mode, serialized=component_serialized)
  File "/usr/local/lib/python3.6/dist-packages/deeppavlov/core/common/params.py", line 104, in from_params
    component = cls(**dict(config_params, **kwargs))
  File "/usr/local/lib/python3.6/dist-packages/deeppavlov/models/embedders/abstract_embedder.py", line 56, in __init__
    self.load()
  File "/usr/local/lib/python3.6/dist-packages/deeppavlov/models/embedders/fasttext_embedder.py", line 68, in load
    self.model = fastText.load_model(str(self.load_path))
  File "/usr/local/lib/python3.6/dist-packages/fastText/FastText.py", line 303, in load_model
    return _FastText(path)
  File "/usr/local/lib/python3.6/dist-packages/fastText/FastText.py", line 37, in __init__
    self.f.loadModel(model)
ValueError: /root/.deeppavlov/downloads/embeddings/lenta_lower_100.bin cannot be opened for loading!
yoptar commented 5 years ago

Looks like you don't have embeddings for this config downloaded. You can run build_model(configs.ner.ner_rus, download=True) to fix that.

bavadim commented 5 years ago

I can't. I don't have internet connection on that server. But I can create distribution on computer with internet. Where can I load lenta_lower_100.bin file?

yoptar commented 5 years ago

Then you can download everything for that model on another computer. For example:

DP_ROOT_PATH='./model_files' python -m deeppavlov download ner_rus

Your model files will be downloaded to the model_files local directory

yoptar commented 5 years ago

I'm assuming the answer helped. Closing because of inactivity.