Electrofried
commented
1 year ago Unfortunatly sentance transformers needs to pull the specified model from there the first time it loads. IF you have loaded it before and already have the model then it should work, but afaik unless you change the transformer model and copy the data to the right location your self there is no way around this.
Josh-XT
Description
Hugging face (HF) is down for the moment, because of huggingface's API dependency it seems that we cannot use Agent-LLM when HF is down. When starting app.py and then launching a new task with oobaboga it fails to start :
INFO: 127.0.0.1:57042 - "POST /api/agent/Vicuna%201.1/task HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 259, in hf_raise_for_status response.raise_for_status() File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/distilbert-base-uncased
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 435, in run_asgi result = await app( # type: ignore[func-returns-value] File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in call return await self.app(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/fastapi/applications.py", line 276, in call await super().call(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/applications.py", line 122, in call await self.middleware_stack(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in call raise exc File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in call await self.app(scope, receive, _send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/middleware/cors.py", line 92, in call await self.simple_response(scope, receive, send, request_headers=headers) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/middleware/cors.py", line 147, in simple_response await self.app(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in call raise exc File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in call await self.app(scope, receive, sender) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in call raise e File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call await self.app(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/routing.py", line 718, in call await route.handle(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle await self.app(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/routing.py", line 66, in app response = await func(request) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/fastapi/routing.py", line 237, in app raw_response = await run_endpoint_function( File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/fastapi/routing.py", line 163, in run_endpoint_function return await dependant.call(*values) File "/home/brice/Agent-LLM/app.py", line 248, in start_task_agent CFG.agent_instances[agent_name] = AgentLLM(agent_name) File "/home/brice/Agent-LLM/AgentLLM.py", line 41, in init embedding_functions.SentenceTransformerEmbeddingFunction( File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/chromadb/utils/embedding_functions.py", line 14, in init self._model = SentenceTransformer(model_name) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 87, in init snapshot_download(model_name_or_path, File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/sentence_transformers/util.py", line 442, in snapshot_download model_info = _api.model_info(repo_id=repo_id, revision=revision, token=token) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 120, in _inner_fn return fn(args, **kwargs) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/huggingface_hub/hf_api.py", line 1604, in model_info hf_raise_for_status(r) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 301, in hf_raise_for_status raise HfHubHTTPError(str(e), response=response) from e huggingface_hub.utils._errors.HfHubHTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/distilbert-base-uncased
Steps to Reproduce the Bug
Fresh local install
Launch app.py > Agent > Task Agent > Start pursuing task
Expected Behavior
Less dependency on third party services and functioning even if huggingface is down.
Actual Behavior
task is not starting, an exception is thrown :
INFO: 127.0.0.1:57042 - "POST /api/agent/Vicuna%201.1/task HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 259, in hf_raise_for_status response.raise_for_status() File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/requests/models.py", line 1021, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/distilbert-base-uncased
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 435, in run_asgi result = await app( # type: ignore[func-returns-value] File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in call return await self.app(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/fastapi/applications.py", line 276, in call await super().call(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/applications.py", line 122, in call await self.middleware_stack(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in call raise exc File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in call await self.app(scope, receive, _send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/middleware/cors.py", line 92, in call await self.simple_response(scope, receive, send, request_headers=headers) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/middleware/cors.py", line 147, in simple_response await self.app(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in call raise exc File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in call await self.app(scope, receive, sender) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in call raise e File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call await self.app(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/routing.py", line 718, in call await route.handle(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle await self.app(scope, receive, send) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/starlette/routing.py", line 66, in app response = await func(request) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/fastapi/routing.py", line 237, in app raw_response = await run_endpoint_function( File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/fastapi/routing.py", line 163, in run_endpoint_function return await dependant.call(*values) File "/home/brice/Agent-LLM/app.py", line 248, in start_task_agent CFG.agent_instances[agent_name] = AgentLLM(agent_name) File "/home/brice/Agent-LLM/AgentLLM.py", line 41, in init embedding_functions.SentenceTransformerEmbeddingFunction( File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/chromadb/utils/embedding_functions.py", line 14, in init self._model = SentenceTransformer(model_name) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 87, in init snapshot_download(model_name_or_path, File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/sentence_transformers/util.py", line 442, in snapshot_download model_info = _api.model_info(repo_id=repo_id, revision=revision, token=token) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 120, in _inner_fn return fn(args, **kwargs) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/huggingface_hub/hf_api.py", line 1604, in model_info hf_raise_for_status(r) File "/home/brice/miniconda3/envs/agent/lib/python3.10/site-packages/huggingface_hub/utils/_errors.py", line 301, in hf_raise_for_status raise HfHubHTTPError(str(e), response=response) from e huggingface_hub.utils._errors.HfHubHTTPError: 504 Server Error: Gateway Time-out for url: https://huggingface.co/api/models/distilbert-base-uncased
Additional Context / Screenshots
No response
Operating System
Python Version
Environment Type - Connection
Environment Type - Container
Acknowledgements