search

thammegowda / nllb-serve

Meta's "No Language Left Behind" models served as web app and REST API
http://rtg.isi.edu/nllb/
183 stars 27 forks source link

NllbTokenizerFast' object has no attribute 'lang_code_to_id #20

Open Devloper-Sigmoid opened 4 months ago

Devloper-Sigmoid commented 4 months ago

NllbTokenizerFast' object has no attribute 'lang_code_to_id

GOvEy1nw commented 4 months ago

hey, I found a fix for this.

line 145 in app.py needs changing from:

**inputs, forced_bos_token_id=tokenizer.lang_code_to_id[tgt_lang],

To:

**inputs, forced_bos_token_id=tokenizer.convert_tokens_to_ids(tgt_lang),

snap-star commented 4 months ago

hey, I found a fix for this.

line 145 in app.py needs changing from:

**inputs, forced_bos_token_id=tokenizer.lang_code_to_id[tgt_lang],

To:

**inputs, forced_bos_token_id=tokenizer.convert_tokens_to_ids(tgt_lang),

thanks a lot man!

brauliobo commented 1 month ago

fixed here by downgrading transformers, see https://github.com/huggingface/transformers/issues/31348#issuecomment-2383895798

jksab commented 2 weeks ago

hey, I found a fix for this.

line 145 in app.py needs changing from:

**inputs, forced_bos_token_id=tokenizer.lang_code_to_id[tgt_lang],

To:

**inputs, forced_bos_token_id=tokenizer.convert_tokens_to_ids(tgt_lang),

Thanks it worked, you fixed it