hi! any hints on how to use it for question answering in spanish?

[juancresc]

I've followed some examples in transformers with no success:

from transformers import AutoTokenizer, TFAutoModelForQuestionAnswering
import tensorflow as tf

tokenizer = AutoTokenizer.from_pretrained("./data/beto/")
model = TFAutoModelForQuestionAnswering.from_pretrained("./data/beto/")

ValueError: Unrecognized model in ./data/beto/. Should have a `model_type` key in its config.json

[juancresc]

I did it till here:

tokenizer = AutoTokenizer.from_pretrained("mrm8488/bert-base-spanish-wwm-cased-finetuned-spa-squad2-es")

model = AutoModelForQuestionAnswering.from_pretrained("mrm8488/bert-base-spanish-wwm-cased-finetuned-spa-squad2-es")

question, text = "Cuanto pesa?", "Jim Henson pesa 10 kilos"
encoding = tokenizer.encode_plus(question, text, return_tensors="pt")

input_ids = encoding["input_ids"]
attention_mask = encoding["attention_mask"]

start_scores, end_scores = model(input_ids, attention_mask=attention_mask)
all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0].tolist())

answer_tokens = all_tokens[torch.argmax(start_scores) :torch.argmax(end_scores)+1]
answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))
answer

but answer is always [CLS]

[juancresc]

If I add a longer text, seems to work decently. Much of the responses are [UNK] but I guess it is a different story