tokenizer failing to load

yousofaly commented 3 months ago

when running

python lhrs_webui.py -c Config/multi_modal_eval.yaml --server-port 8000 --server-name 127.0.0.1 --share

I see this error

/home/.pyenv/versions/lhrs/lib/python3.10/site-packages/huggingface_hub/file_download.py:1150: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:03<00:00,  1.73s/it]
Traceback (most recent call last):
  File "/home/yousof/LHRS-Bot/lhrs_webui.py", line 917, in <module>
    model, tokenizer = _load_model_tokenizer(args)
  File "/home/yousof/LHRS-Bot/lhrs_webui.py", line 107, in _load_model_tokenizer
    return model, tokenizer
UnboundLocalError: local variable 'tokenizer' referenced before assignment

That's because I am trying to run the demo with the default checkpoint path path by leaving it blank. But leaving it blank will result in the if statement on line 88 here to not run.

how can I run the demo by automatically downloading the models from hugging face?

pUmpKin-Co commented 3 months ago

Hi,

Please copy tokenizer = model.text.tokenizer from line 103 to line 104, ensuring the correct indentation. However, we recommend using the checkpoint for chat, as not loading the checkpoint may result in meaningless output.

yousofaly commented 3 months ago

per the readme, the checkpoints are being automatically downloaded from huggingface, correct?

NJU-LHRS / LHRS-Bot

tokenizer failing to load #23