thu-coai / ConvLab-2

ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems
Apache License 2.0
449 stars 130 forks source link

[Maintenance] BERTNLU initialisation fails #203

Closed alexcoca closed 3 years ago

alexcoca commented 3 years ago

Describe the feature BERTNLU class should check if the spacy language model is available before calling spacy.load in L59. Otherwise, a user that is not acquainted with the model internals and spacy runs into the following error:

OSError: [E050] Can't find model 'en_core_web_sm'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.
#4577

The solution is something along the lines of:

try:
    self.nlp = spacy.load("en_core_web_sm")
except OSError:
    spacy_download("en_core_web_sm")
    from spacy.cli import link
    from spacy.util import get_package_path
    package_path = get_package_path("en_core_web_sm")
    link(spacy_model_name, "en_core_web_sm", model_path=package_path)
    self.nlp = spacy.load("en_core_web_sm", disable=disable)

which is the approach taken in allennlp.common.util.py.

Expected behavior Users do not get OSError when using parts of the library that make use of spacy models that are not installed/downloaded.

zqwerty commented 3 years ago

That's good! We will fix it