Fail to load model archive after training

[titipata]

Hi all, I followed an example and finally got the trained model saved in model.tar.gz. I want to load the model to make a predictor like following

from allennlp.models.archival import load_archive
from allennlp.service.predictors import Predictor

archive = load_archive('model.tar.gz')
# predictor = Predictor.from_archive(archive, 'paper-classifier')

However, I got the following error when I try to load the model.

ConfigurationError                        Traceback (most recent call last)
<ipython-input-2-741f7f19114e> in <module>()
----> 1 archive = load_archive('output/model.tar.gz')

~/anaconda3/lib/python3.6/site-packages/allennlp/models/archival.py in load_archive(archive_file, cuda_device, overrides, weights_file)
    147                        weights_file=weights_path,
    148                        serialization_dir=serialization_dir,
--> 149                        cuda_device=cuda_device)
    150 
    151     if tempdir:

~/anaconda3/lib/python3.6/site-packages/allennlp/models/model.py in load(cls, config, serialization_dir, weights_file, cuda_device)
    293         # This allows subclasses of Model to override _load.
    294         # pylint: disable=protected-access
--> 295         return cls.by_name(model_type)._load(config, serialization_dir, weights_file, cuda_device)
    296 
    297 

~/anaconda3/lib/python3.6/site-packages/allennlp/common/registrable.py in by_name(cls, name)
     54     def by_name(cls: Type[T], name: str) -> Type[T]:
     55         if name not in Registrable._registry[cls]:
---> 56             raise ConfigurationError("%s is not a registered name for %s" % (name, cls.__name__))
     57         return Registrable._registry[cls].get(name)
     58 

ConfigurationError: 'paper-classifier is not a registered name for Model'

Is their a way on how to register paper-classifier so that it can be loaded and use later?

[matt-gardner]

Looks like you're doing this in your own script instead of using our command line tools, right? In that case, just make sure you import your classes into the script; when python imports the class, there is a decorator that registers it. If you're using our command line tools, this is done with the --include-package flag. In your own script, just import your classes, and you'll be good.

[titipata]

Thanks for the quick reply @matt-gardner! Yeah, you're right, I'd like to do this in own script instead of using command line tools.

As you suggested, in this example case if I want to use it, it seems like I have to do the following:

from my_library.predictors import PaperClassifierPredictor
from my_library.dataset_readers import SemanticScholarDatasetReader
from my_library.models import AcademicPaperClassifier

dataset_reader = SemanticScholarDatasetReader()
model = AcademicPaperClassifier(...)
predictor = PaperClassifierPredictor(model, dataset_reader)

Is there an easier way to use load_archive then use the trained model right away (the same way as those registered one e.g. 'constituency-parser')?

[matt-gardner]

You can keep your original code as it is, just add imports at the beginning:

# if you set up your __init__.py right, you can just do `import my_library` instead of these three lines.
from my_library.predictors import PaperClassifierPredictor
from my_library.dataset_readers import SemanticScholarDatasetReader
from my_library.models import AcademicPaperClassifier

# Now that we've imported the classes, they have been registered, and we can proceed with
# your original code.

from allennlp.models.archival import load_archive
from allennlp.service.predictors import Predictor

archive = load_archive('model.tar.gz')
predictor = Predictor.from_archive(archive, 'paper_classifier')  # this should now work

[titipata]

Ah, I got it now @matt-gardner. This works like a charm! I will close the issue for now.

It would be great if this can be found somewhere in the documentation though.

Note that the model name in the example is paper-classifier so I changed the name in my question.

Example to predict:

import json
lines = open('tests/fixtures/s2_papers.jsonl', 'r').readlines()
output = [predictor.predict_json(json.loads(line)) for line in lines]