NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE.

[efraim-evo]

Hello, I'm trying to use the demo on Google Colab but it's giving me this error when I enter a question, can anyone help me? Here is the notebook: https://colab.research.google.com/drive/1E-qCUuQv9hGErLGe5LEfaOuFG6V7bqeU?usp=sharing

[chrisjuniorli]

you can try the demo here: https://huggingface.co/spaces/shi-labs/CuMo-7b-zero, it seems the repo id you used is invalid: requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/checkpoints/CuMo-mistral-7b/resolve/main/tokenizer_config.json

[efraim-evo]

Can you send me a colab notebook solving this problem? I'm trying to test on Google Colab to run the application on a real machine.

[chricro]

I get the same error on the gradio interface when running the following command:

CUDA_VISIBLE_DEVICES=0 python -m cumo.serve.app \
    --model-path checkpoints/CuMo-mistral-7b

Internally, I get this error: NameError: name 'LlavaMixtralForCausalLM' is not defined

[chrisjuniorli]

@chricro can you share more error report info here, like in which line or fucntion so that I can help debug it

[chricro]

Sure, here is the error I got:

CUDA_VISIBLE_DEVICES=0,1,2,3 python -m cumo.serve.app --model-path shi-labs/CuMo-mistral-7b [2024-06-12 13:32:11,400] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to cuda (auto detect) 2024-06-12 13:32:13 | INFO | stdout | args: Namespace(host='0.0.0.0', port=None, model_path='shi-labs/CuMo-mistral-7b', model_base=None, controller_url='http://localhost:10000', concurrency_count=5, bits=16, model_list_mode='reload', share=False, moderate=False, embed=False) 2024-06-12 13:32:13 | INFO | stdout | Starting the controller 2024-06-12 13:32:13 | INFO | stdout | ['/home/paperspace/cumo_test_2/CuMo/venv/bin/python', '-m', 'cumo.serve.controller', '--host', '0.0.0.0', '--port', '10000'] 2024-06-12 13:32:13 | INFO | stdout | Starting the model worker for the model shi-labs/CuMo-mistral-7b 2024-06-12 13:32:13 | INFO | stdout | ['/home/paperspace/cumo_test_2/CuMo/venv/bin/python', '-m', 'cumo.serve.model_worker', '--host', '0.0.0.0', '--controller', 'http://localhost:10000', '--model-path', 'shi-labs/CuMo-mistral-7b', '--model-name', 'CuMo-mistral-7b', '--use-flash-attn'] [2024-06-12 13:32:16,929] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to cuda (auto detect) [2024-06-12 13:32:16,952] [INFO] [real_accelerator.py:191:get_accelerator] Setting ds_accelerator to cuda (auto detect) 2024-06-12 13:32:17 | INFO | model_worker | args: Namespace(host='0.0.0.0', port=21002, worker_address='http://localhost:21002', controller_address='http://localhost:10000', model_path='shi-labs/CuMo-mistral-7b', model_base=None, model_name='CuMo-mistral-7b', device='cuda', multi_modal=False, limit_model_concurrency=5, stream_interval=1, no_register=False, load_8bit=False, load_4bit=False, use_flash_attn=True) 2024-06-12 13:32:17 | INFO | model_worker | Loading the model CuMo-mistral-7b on worker ce9117 ... 2024-06-12 13:32:17 | INFO | stdout | Loading from full model... 2024-06-12 13:32:17 | INFO | stdout | Loading CuMo 7b model... 2024-06-12 13:32:17 | ERROR | stderr | /home/paperspace/cumo_test_2/CuMo/venv/lib/python3.11/site-packages/huggingface_hub/file_download.py:1132: FutureWarning:resume_downloadis deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, useforce_download=True. 2024-06-12 13:32:17 | ERROR | stderr | warnings.warn( 2024-06-12 13:32:17 | INFO | controller | args: Namespace(host='0.0.0.0', port=10000, dispatch_method='shortest_queue') 2024-06-12 13:32:17 | INFO | controller | Init controller 2024-06-12 13:32:17 | ERROR | stderr | INFO: Started server process [32460] 2024-06-12 13:32:17 | ERROR | stderr | INFO: Waiting for application startup. 2024-06-12 13:32:17 | ERROR | stderr | INFO: Application startup complete. 2024-06-12 13:32:17 | ERROR | stderr | INFO: Uvicorn running on http://0.0.0.0:10000 (Press CTRL+C to quit) 2024-06-12 13:32:18 | ERROR | stderr | Traceback (most recent call last): 2024-06-12 13:32:18 | ERROR | stderr | File "<frozen runpy>", line 198, in _run_module_as_main 2024-06-12 13:32:18 | ERROR | stderr | File "<frozen runpy>", line 88, in _run_code 2024-06-12 13:32:18 | ERROR | stderr | File "/home/paperspace/cumo_test_2/CuMo/cumo/serve/model_worker.py", line 280, in <module> 2024-06-12 13:32:18 | ERROR | stderr | worker = ModelWorker(args.controller_address, 2024-06-12 13:32:18 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-12 13:32:18 | ERROR | stderr | File "/home/paperspace/cumo_test_2/CuMo/cumo/serve/model_worker.py", line 65, in __init__ 2024-06-12 13:32:18 | ERROR | stderr | self.tokenizer, self.model, self.image_processor, self.context_len = load_pretrained_model( 2024-06-12 13:32:18 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^ 2024-06-12 13:32:18 | ERROR | stderr | File "/home/paperspace/cumo_test_2/CuMo/cumo/model/builder.py", line 127, in load_pretrained_model 2024-06-12 13:32:18 | ERROR | stderr | model = LlavaMistralForCausalLM.from_pretrained( 2024-06-12 13:32:18 | ERROR | stderr | ^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-12 13:32:18 | ERROR | stderr | NameError: name 'LlavaMistralForCausalLM' is not defined 2024-06-12 13:32:24 | ERROR | stderr | /home/paperspace/cumo_test_2/CuMo/venv/lib/python3.11/site-packages/gradio/components/dropdown.py:163: UserWarning: The value passed into gr.Dropdown() is not in the list of choices. Please update the list of choices to include: or set allow_custom_value=True. 2024-06-12 13:32:24 | ERROR | stderr | warnings.warn( 2024-06-12 13:32:24 | INFO | stdout | Running on local URL: http://0.0.0.0:7860 2024-06-12 13:32:24 | INFO | stdout | 2024-06-12 13:32:24 | INFO | stdout | To create a public link, setshare=Trueinlaunch(). 2024-06-12 13:32:24 | INFO | stdout | IMPORTANT: You are using gradio version 4.16.0, however version 4.29.0 is available, please upgrade. 2024-06-12 13:32:24 | INFO | stdout | --------

[chrisjuniorli]

it seems the LlavaMistralForCausalLM was not found here, can you try from cumo.model import * alone in a python script or interactive mode and see if it reports error

[chricro]

my bad @chrisjuniorli, I was using python 3.11 and downgrading to 3.9 solved the issue for me