search

danswer-ai / danswer

Gen-AI Chat for Teams - Think ChatGPT if it had access to your team's unique knowledge.
https://danswer.ai
Other
10.65k stars 1.33k forks source link

api_server can not start: "safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer" #1123

Open luqijun opened 8 months ago

luqijun commented 8 months ago 
2024-02-23 15:51:23 Starting Danswer Api Server
2024-02-23 16:01:40 Starting Danswer Api Server
2024-02-23 15:51:21 INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
2024-02-23 15:51:21 INFO  [alembic.runtime.migration] Will assume transactional DDL.
2024-02-23 15:51:31 INFO:     Started server process [21]
2024-02-23 15:51:31 INFO:     Waiting for application startup.
2024-02-23 15:51:31 02/23/2024 07:51:31 AM             users.py  69 : Using Auth Type: disabled
2024-02-23 15:51:31 02/23/2024 07:51:31 AM              main.py 235 : Danswer API Key: dn_ZcxhXaQzxoUY9z0bAI6ySA2MONCz4R4mMVU_zQHX
2024-02-23 15:51:31 02/23/2024 07:51:31 AM              main.py 243 : Using LLM Provider: openai
2024-02-23 15:51:31 02/23/2024 07:51:31 AM              main.py 245 : Using LLM Model Version: gpt-3.5-turbo-16k-0613
2024-02-23 15:51:31 02/23/2024 07:51:31 AM          chat_llm.py  49 : LLM Model Class: ChatLiteLLM, Model Config: {'model': 'openai/gpt-3.5-turbo-16k-0613', 'request_timeout': 60.0, 'model_kwargs': {'frequency_penalty': 0, 'presence_penalty': 0}, 'n': 1, 'max_tokens': 1024}
2024-02-23 15:51:32 02/23/2024 07:51:32 AM              main.py 275 : Using Embedding model: "intfloat/e5-base-v2"
2024-02-23 15:51:32 02/23/2024 07:51:32 AM              main.py 277 : Query embedding prefix: ""
2024-02-23 15:51:32 02/23/2024 07:51:32 AM              main.py 280 : Passage embedding prefix: "passage: "
2024-02-23 15:51:32 02/23/2024 07:51:32 AM              main.py 292 : Warming up local NLP models.
2024-02-23 15:51:32 /usr/local/lib/python3.11/site-packages/transformers/utils/hub.py:128: FutureWarning: Using `DISABLE_TELEMETRY` is deprecated and will be removed in v5 of Transformers. Use `HF_HUB_DISABLE_TELEMETRY` instead.
2024-02-23 15:51:32   warnings.warn(
2024-02-23 15:51:33 02/23/2024 07:51:33 AM search_nlp_models.py 103 : Loading intfloat/e5-base-v2
2024-02-23 15:51:33 ERROR:    Traceback (most recent call last):
2024-02-23 15:51:33   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 734, in lifespan
2024-02-23 15:51:33     async with self.lifespan_context(app) as maybe_state:
2024-02-23 15:51:33   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 610, in __aenter__
2024-02-23 15:51:33     await self._router.startup()
2024-02-23 15:51:33   File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 713, in startup
2024-02-23 15:51:33     handler()
2024-02-23 15:51:33   File "/app/danswer/main.py", line 293, in startup_event
2024-02-23 15:51:33     warm_up_models(
2024-02-23 15:51:33   File "/app/danswer/search/search_nlp_models.py", line 383, in warm_up_models
2024-02-23 15:51:33     embed_model.encode(texts=[warm_up_str], text_type=EmbedTextType.QUERY)
2024-02-23 15:51:33   File "/app/danswer/search/search_nlp_models.py", line 232, in encode
2024-02-23 15:51:33     local_model = self.load_model()
2024-02-23 15:51:33                   ^^^^^^^^^^^^^^^^^
2024-02-23 15:51:33   File "/app/danswer/search/search_nlp_models.py", line 202, in load_model
2024-02-23 15:51:33     return get_local_embedding_model(
2024-02-23 15:51:33            ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-23 15:51:33   File "/app/danswer/search/search_nlp_models.py", line 104, in get_local_embedding_model
2024-02-23 15:51:33     _EMBED_MODEL = (SentenceTransformer(model_name), model_name)
2024-02-23 15:51:33                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-23 15:51:33   File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 95, in __init__
2024-02-23 15:51:33     modules = self._load_sbert_model(model_path)
2024-02-23 15:51:33               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-23 15:51:33   File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 840, in _load_sbert_model
2024-02-23 15:51:33     module = module_class.load(os.path.join(model_path, module_config['path']))
2024-02-23 15:51:33              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-23 15:51:33   File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 137, in load
2024-02-23 15:51:33     return Transformer(model_name_or_path=input_path, **config)
2024-02-23 15:51:33            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-23 15:51:33   File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 29, in __init__
2024-02-23 15:51:33     self._load_model(model_name_or_path, config, cache_dir)
2024-02-23 15:51:33   File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 49, in _load_model
2024-02-23 15:51:33     self.auto_model = AutoModel.from_pretrained(model_name_or_path, config=config, cache_dir=cache_dir)
2024-02-23 15:51:33                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-23 15:51:33   File "/usr/local/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
2024-02-23 15:51:33     return model_class.from_pretrained(
2024-02-23 15:51:33            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-23 15:51:33   File "/usr/local/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3371, in from_pretrained
2024-02-23 15:51:33     with safe_open(resolved_archive_file, framework="pt") as f:
2024-02-23 15:51:33          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-02-23 15:51:33 safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer
2024-02-23 15:51:33 
2024-02-23 15:51:33 ERROR:    Application startup failed. Exiting.
IgorianKiselev02 commented 8 months ago

Got the same problem - does any one have any ideas?

EilamGoldenberg commented 2 weeks ago

Same here. Might be relevent to mention I trained my (vision transformer) model on a distant GPU, and now want to use it for inference locally. Stops at this command: fine_tuned_model = AutoModelForImageClassification.from_pretrained(results_dir_suffix+"_model0").to(device)