AutoModelForCausalLM doesn't generate as expected

[alonsosilvaallende]

First of all, thank you for this lovely package. I'm currently finding the following error: When I try a basic example translated to transformers-js-py:

from transformers_js_py import import_transformers_js
transformers = await import_transformers_js()

model_id = 'Xenova/distilgpt2'
AutoTokenizer = transformers.AutoTokenizer
tokenizer = await AutoTokenizer.from_pretrained(model_id)
inputs = tokenizer('I love transformers!')

AutoModelForCausalLM = transformers.AutoModelForCausalLM
model = await AutoModelForCausalLM.from_pretrained(model_id)
output = await model.generate(**inputs)

I obtain the following error: JsException: Error: inputs must be a Tensor, TypedArray, or Array, but is "Object".

Any hint on how to make it work? I have tried different conversions but I've been unable to make it work. Thank you very much.

[whitphx]

Hi, sorry for my late reply. Looks like this part

model = await AutoModelForCausalLM.from_pretrained(model_id)
output = await model.generate(**inputs)

should be like this

model = await AutoModelForCausalLM.from_pretrained(model_id)
output = await model.generate(inputs["input_ids"])

as the second example of the doc?

Anyway the error log is little helpful for debugging. I will improve it. Seems like the current stack trace has enough info and it can't be improved?

[alonsosilvaallende]

Thank you very much. This solves this problem. I'm not sure I understand your last message but I do think that either the transformers-js-py documentation or the error log could be more explicit.