search

mbzuai-oryx / Video-ChatGPT

[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.
https://mbzuai-oryx.github.io/Video-ChatGPT
Creative Commons Attribution 4.0 International
1.17k stars 102 forks source link

error while deploying video-chatgpt locally #33

Closed Xinxinatg closed 1 year ago

Xinxinatg commented 1 year ago

I followed the instruction of how to set up the model inference locally, now the server is up but when I uploaded the test video for inference, I was given the error:

2023-07-25 08:26:23 | ERROR | asyncio | Task exception was never retrieved future: <Task finished name='7rg4m1ee7gf_12' coro=<Queue.process_events() done, defined at /home/conducivedev/.conda/envs/video_chatgpt/lib/python3.10/site-packages/gradio/queueing.py:343> exception=1 validation error for PredictBody event_id Field required [type=missing, input_value={'fn_index': 12, 'data': ...on_hash': '7rg4m1ee7gf'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.0.3/v/missing> Traceback (most recent call last): File "/home/conducivedev/.conda/envs/video_chatgpt/lib/python3.10/site-packages/gradio/queueing.py", line 347, in process_events client_awake = await self.gather_event_data(event) File "/home/conducivedev/.conda/envs/video_chatgpt/lib/python3.10/site-packages/gradio/queueing.py", line 220, in gather_event_data data, client_awake = await self.get_message(event, timeout=receive_timeout) File "/home/conducivedev/.conda/envs/video_chatgpt/lib/python3.10/site-packages/gradio/queueing.py", line 456, in get_message return PredictBody(**data), True File "/home/conducivedev/.conda/envs/video_chatgpt/lib/python3.10/site-packages/pydantic/main.py", line 150, in init pydantic_self.pydantic_validator.validate_python(data, self_instance=__pydantic_self__) pydantic_core._pydantic_core.ValidationError: 1 validation error for PredictBody event_id Field required [type=missing, input_value={'fn_index': 12, 'data': ...on_hash': '7rg4m1ee7gf'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.0.3/v/missing Screenshot from 2023-07-25 08-27-16

mmaaz60 commented 1 year ago

Hi @Xinxinatg,

Thank you for your interest in our work. Please try installing the below version of pydantic and let me know if it solves the issue. Thanks.

pip install pydantic==1.10.7
Xinxinatg commented 1 year ago

Thanks for the reply! after installing pydantic package, now I received this error:

OSError: Unable to load weights from pytorch checkpoint file for 'LLaVA-7B-Lightening-v1-1/pytorch_model-00001-of-00002.bin' at 'LLaVA-7B-Lightening-v1-1/pytorch_model-00001-of-00002.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

image

mmaaz60 commented 1 year ago

Hi @Xinxinatg,

What version of transformers library are you using? Please try installing transformers using the following command and let me know if it solves the issue. Thank You.

pip install transformers@git+https://github.com/huggingface/transformers.git@cae78c46
Xinxinatg commented 1 year ago

I think it is the same as in the requirements.txt? I reistalled the transformer as per the insructions, but the error still persist

mmaaz60 commented 1 year ago

Hi @Xinxinatg,

Most probably then the LLaVA-7B-Lightening-v1-1/pytorch_model-00001-of-00002.bin is corrupted. Could you please try downloading the weights again using the below commands and let me know if it works? Thanks

rm -rf LLaVA-7B-Lightening-v1-1
git lfs install
git clone https://huggingface.co/mmaaz60/LLaVA-7B-Lightening-v1-1
Xinxinatg commented 1 year ago

Thanks a lot, it works!