kssteven418 / BigLittleDecoder

[NeurIPS'23] Speculative Decoding with Big Little Decoder
https://arxiv.org/pdf/2302.07863.pdf
Apache License 2.0
85 stars 10 forks source link

How to import T5_BiLD model in run_translation task. #3

Open sufeidechabei opened 1 year ago

sufeidechabei commented 1 year ago

System Info

I try from transformers.models.t5.modeling_t5 import T5_BiLDModel, but it doesn't work. I build the library from transformer repo.

Who can help?

No response

Information

Tasks

Reproduction

Just run the translation task. CUDA_VISIBLE_DEVICES=0 python run_bild_translation.py --model bild --small /nobackup/haozhang/BigLittleDecoder/models/smallmodel --large /nobackup/haozhang/BigLittleDecoder/models/bigmodel \ --dataset_name iwslt2017 --dataset_config iwslt2017-de-en --source_lang de --target_lang en --bild_rollback 3 --bild_fallback 3

Expected behavior

It can not import the T5BiLDModel

hyunW3 commented 4 days ago

It seems that this is mentioned in the following comment: https://github.com/kssteven418/BigLittleDecoder/issues/6#issuecomment-2127891521, specifically in reference to the command python -m pip install -e .