Closed code-snail closed 1 month ago
@fangyinc have you ever encountered this problem before?
I also encountered the same problem when i want to deploy a model named sqlcoder-7b-2, is there any progress so far?
I also encountered the same problem when i want to deploy a model named sqlcoder-7b-2, is there any progress so far?
I manually installed a lower version of the transformers library, which resolved the error. You can do this by running: pip install transformers==4.37
.
I also encountered the same problem when i want to deploy a model named sqlcoder-7b-2, is there any progress so far?
I manually installed a lower version of the transformers library, which resolved the error. You can do this by running:
pip install transformers==4.37
. May I ask if DB-GPT.0.5.0 version can support the sqlcoder-7b-2 model, right? What is the effect of text-2-SQL? @zyan-repository
This issue has been marked as stale
, because it has been over 30 days without any activity.
This issue bas been closed, because it has been marked as stale
and there has been no activity for over 7 days.
Search before asking
Operating system information
Linux
Python version information
DB-GPT version
latest release
Related scenes
Installation Information
[ ] Installation From Source
[X] Docker Installation
[ ] Docker Compose Installation
[ ] Cluster Installation
[ ] AutoDL Image
[ ] Other
Device information
Models information
LLM: vicuna-7b-v1.5
What happened
Model Unified Deployment Mode! Warning: Loading base MPT model with.startup_event..start_worker_manager() done, defined at /app/dbgpt/model/cluster/worker/manager.py:836> exception=SystemExit(1)>
Traceback (most recent call last):
File "/app/dbgpt/model/cluster/worker/manager.py", line 838, in start_worker_manager
await worker_manager.start()
File "/app/dbgpt/model/cluster/worker/manager.py", line 623, in start
return await self.worker_manager.start()
File "/app/dbgpt/model/cluster/worker/manager.py", line 109, in start
raise Exception(out.message)
Exception: model vicuna-7b-v1.5@huggingface(172.17.0.7:5000) start failed, 'LlamaRotaryEmbedding' object has no attribute 'cos_cached'
zero_shot
conversation configuration. If this is not desired, inspect model configurations and names. Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]/usr/local/lib/python3.10/dist-packages/torch/_utils.py:776: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.get(instance, owner)() Loading checkpoint shards: 100%|██████████| 2/2 [00:05<00:00, 2.52s/it] /usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:410: UserWarning:do_sample
is set toFalse
. However,temperature
is set to0.9
-- this flag is only used in sample-based generation modes. You should setdo_sample=True
or unsettemperature
. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed. warnings.warn( /usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:415: UserWarning:do_sample
is set toFalse
. However,top_p
is set to0.6
-- this flag is only used in sample-based generation modes. You should setdo_sample=True
or unsettop_p
. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed. warnings.warn( /usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:410: UserWarning:do_sample
is set toFalse
. However,temperature
is set to0.9
-- this flag is only used in sample-based generation modes. You should setdo_sample=True
or unsettemperature
. warnings.warn( /usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:415: UserWarning:do_sample
is set toFalse
. However,top_p
is set to0.6
-- this flag is only used in sample-based generation modes. You should setdo_sample=True
or unsettop_p
. warnings.warn( /usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py:4193: FutureWarning:_is_quantized_training_enabled
is going to be deprecated in transformers 4.39.0. Please usemodel.hf_quantizer.is_trainable
instead warnings.warn( The cos_cached attribute will be removed in 4.40. Bear in mind that its contents changed in v4.38. Use the forward method of RoPE from now on instead. 2024-02-29 09:22:36 d0c761fd01bc dbgpt.model.cluster.worker.manager[1] ERROR Error starting worker manager: model vicuna-7b-v1.5@huggingface(172.17.0.7:5000) start failed, 'LlamaRotaryEmbedding' object has no attribute 'cos_cached' 2024-02-29 09:22:36 d0c761fd01bc asyncio[1] ERROR Task exception was never retrieved future: <Task finished name='Task-4' coro=<_setup_fastapi.During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "uvloop/loop.pyx", line 474, in uvloop.loop.Loop._on_idle File "uvloop/cbhandles.pyx", line 83, in uvloop.loop.Handle._run File "uvloop/cbhandles.pyx", line 63, in uvloop.loop.Handle._run File "/app/dbgpt/model/cluster/worker/manager.py", line 841, in start_worker_manager sys.exit(1) SystemExit: 1 2024-02-29 09:22:36 d0c761fd01bc dbgpt.model.cluster.worker.manager[1] INFO Stop all workers 2024-02-29 09:22:36 d0c761fd01bc dbgpt.model.cluster.worker.manager[1] INFO Apply req: None, apply_func: <function LocalWorkerManager._stop_all_worker.._stop_worker at 0x7fe3cc5755a0>
2024-02-29 09:22:36 d0c761fd01bc dbgpt.model.cluster.worker.manager[1] INFO Apply to all workers
2024-02-29 09:22:36 d0c761fd01bc dbgpt.model.cluster.worker.default_worker[1] WARNING Model has been stopped!!
INFO: 127.0.0.1:54858 - "DELETE /api/controller/models?model_name=WorkerManager%40service&host=172.17.0.7&port=5000&weight=1.0&check_healthy=true&healthy=false&enabled=true&prompt_template=&last_heartbeat= HTTP/1.1" 200 OK
INFO: 127.0.0.1:54860 - "DELETE /api/controller/models?model_name=vicuna-7b-v1.5%40llm&host=172.17.0.7&port=5000&weight=1.0&check_healthy=true&healthy=false&enabled=true&prompt_template=&last_heartbeat= HTTP/1.1" 200 OK
ERROR: Traceback (most recent call last):
File "/app/dbgpt/model/cluster/worker/manager.py", line 838, in start_worker_manager
await worker_manager.start()
File "/app/dbgpt/model/cluster/worker/manager.py", line 623, in start
return await self.worker_manager.start()
File "/app/dbgpt/model/cluster/worker/manager.py", line 109, in start
raise Exception(out.message)
Exception: model vicuna-7b-v1.5@huggingface(172.17.0.7:5000) start failed, 'LlamaRotaryEmbedding' object has no attribute 'cos_cached'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "uvloop/loop.pyx", line 1511, in uvloop.loop.Loop.run_until_complete File "uvloop/loop.pyx", line 1504, in uvloop.loop.Loop.run_until_complete File "uvloop/loop.pyx", line 1377, in uvloop.loop.Loop.run_forever File "uvloop/loop.pyx", line 555, in uvloop.loop.Loop._run File "uvloop/loop.pyx", line 474, in uvloop.loop.Loop._on_idle File "uvloop/cbhandles.pyx", line 83, in uvloop.loop.Handle._run File "uvloop/cbhandles.pyx", line 63, in uvloop.loop.Handle._run File "/app/dbgpt/model/cluster/worker/manager.py", line 841, in start_worker_manager sys.exit(1) SystemExit: 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 686, in lifespan await receive() File "/usr/local/lib/python3.10/dist-packages/uvicorn/lifespan/on.py", line 137, in receive return await self.receive_queue.get() File "/usr/lib/python3.10/asyncio/queues.py", line 159, in get await getter asyncio.exceptions.CancelledError
What you expected to happen
docker container start failed with logs
How to reproduce
use docker container
Additional context
No response
Are you willing to submit PR?