Timoeller
commented
3 years ago Hey sorry for the late reply, I actually had to dig into the code a bit. So there are two ways to load models.
Loading model with internet access
The easiest is just to load a QAInferencer with your model name supplied. Then it downloads and converts everything you need. You can do that with:
infer = QAInferencer.load(model_name_or_path="wptoux/albert-chinese-large-qa", task_type="question_answering", gpu=True)
Then you can save and load that Inferencer with .save(foo/bar) and .load(foo/bar)
Loading transformers model saved locally
If your transformers model happens to be saved locally (e.g. with git lfs cloning) and you do not have internet access at that machine you have to construct the FARM Reader model a bit. Here is the code for it:
from farm.modeling.adaptive_model import AdaptiveModel
from farm.utils import initialize_device_settings
from farm.infer import Inferencer, QAInferencer
from farm.data_handler.processor import SquadProcessor
from farm.modeling.tokenization import Tokenizer
from pathlib import Path
device, n_gpu = initialize_device_settings(use_cuda=True)
model_name = "local_models/xlm-roberta-base-squad2"
# Load a Tokenizer
tokenizer = Tokenizer.load(
pretrained_model_name_or_path=model_name
)
# stick it into a Processor
label_list = ["start_token", "end_token"]
processor = SquadProcessor(
tokenizer=tokenizer,
max_seq_len=256,
label_list=label_list,
data_dir=Path("../data/squad20"),
)
# Convert the local transformers model to FARM style
model = AdaptiveModel.convert_from_transformers(model_name,device=device,task_type="question_answering")
# Put everything into a Inferencer - at this point you have a ReaderModel
infer = Inferencer(model=model,processor=processor,task_type="question_answering")
# Testing the Reader Model
QA_input = [
{
"questions": ["Who counted the game among the best ever made?"],
"text": "Twilight Princess was released to universal critical acclaim and commercial success. It received perfect scores from major publications such as 1UP.com, Computer and Video Games, Electronic Gaming Monthly, Game Informer, GamesRadar, and GameSpy. On the review aggregators GameRankings and Metacritic, Twilight Princess has average scores of 95% and 95 for the Wii version and scores of 95% and 96 for the GameCube version. GameTrailers in their review called it one of the greatest games ever created."
}]
print(infer.inference_from_dicts(dicts=QA_input)[0])Btw what are you using this model for? Are you using it in haystack?
datalee
model like thishttps://huggingface.co/wptoux/albert-chinese-large-qa
Edit Timo: So the problem was accessing the HF modelhub from China. There you must add a mirror
as datalee pointed out below