search

eosphoros-ai / DB-GPT

AI Native Data App Development framework with AWEL(Agentic Workflow Expression Language) and Agents
http://docs.dbgpt.cn
MIT License
13.31k stars 1.77k forks source link

[Bug] [Module Name] run python ./dbgpt/app/dbgpt_server.py error 2024-06-11 11:29:00 gptai dbgpt.model.cluster.worker.manager[15737] ERROR Error starting worker manager: model text2vec@huggingface(10.0.18.15:6006) start failed for network error, please make sure your port is available, if you are using global network proxy, please close it #1621

Closed chuangzhidan closed 3 months ago

chuangzhidan commented 3 months ago

Search before asking

Operating system information

Linux

Python version information

=3.11

DB-GPT version

main

Related scenes

Installation Information

Device information

GPU count:2*A100 | NVIDIA-SMI 535.54.03 Driver Version: 535.54.03 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA A800 80GB PCIe Off | 00000000:17:00.0 Off | 0 | | N/A 36C P0 66W / 300W | 8541MiB / 81920MiB | 0% Default | | | | Disabled | +-----------------------------------------+----------------------+----------------------+ | 1 NVIDIA A800 80GB PCIe Off | 00000000:31:00.0 Off | 0 | | N/A 34C P0 66W / 300W | 1802MiB / 81920MiB | 0% Default |

Models information

(dbgpt_env) root@gptai:/media/data/xgp/repo/DB-GPT# python ./dbgpt/app/dbgpt_server.py --host 10.0.18.15 --port 6006 LLM_MODEL: text2vec MODEL_SERVER: http://10.0.18.15:6006 ROOT_PATH:/media/data/xgp/repo/DB-GPT

**=========================== WebServerParameters ===========================

host: 10.0.18.15 port: 6006 daemon: False controller_addr: None model_name: None share: False remote_embedding: False remote_rerank: False log_level: None light: False log_file: dbgpt_webserver.log tracer_file: dbgpt_webserver_tracer.jsonl** tracer_storage_cls: None disable_alembic_upgrade: False awel_dirs: None default_thread_pool_size: None

=========================== WebServerParameters ===========================

host: 10.0.18.15 port: 6006 daemon: False controller_addr: None model_name: text2vec share: False remote_embedding: False remote_rerank: False log_level: INFO light: False log_file: dbgpt_webserver.log tracer_file: dbgpt_webserver_tracer.jsonl tracer_storage_cls: None disable_alembic_upgrade: False awel_dirs: None default_thread_pool_size: None

======================================================================

param:

=========================== WebServerParameters ===========================

host: 10.0.18.15 port: 6006 daemon: False controller_addr: None model_name: text2vec share: False remote_embedding: False remote_rerank: False log_level: INFO light: False log_file: dbgpt_webserver.log tracer_file: dbgpt_webserver_tracer.jsonl tracer_storage_cls: None disable_alembic_upgrade: False awel_dirs: None default_thread_pool_size: None

======================================================================

,param.port:6006 param.port:6006 ------- text2vec ------- /media/data/xgp/repo/DB-GPT/models/text2vec-large-chinese 以上为中间打印内容 2024-06-11 11:24:10 gptai dbgpt.component[15737] INFO Register component with name dbgpt_unified_metadata_db_manager_factory and instance: <dbgpt.storage.metadata.db_factory.UnifiedDBManagerFactory object at 0x7f49689a5c90> /root/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/pydantic/_internal/_fields.py:160: UserWarning: Field "modelname" has conflict with protected namespace "model".

You may be able to resolve this warning by setting model_config['protected_namespaces'] = (). warnings.warn( /root/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/pydantic/_internal/_fields.py:160: UserWarning: Field "modelname" has conflict with protected namespace "model".

You may be able to resolve this warning by setting model_config['protected_namespaces'] = (). warnings.warn( 2024-06-11 11:24:25 gptai dbgpt.component[15737] INFO Register component with name dbgpt_thread_pool_default and instance: <dbgpt.util.executor_utils.DefaultExecutorFactory object at 0x7f4945bdbbb0> 2024-06-11 11:24:25 gptai dbgpt.component[15737] INFO Register component with name dbgpt_default_scheduler and instance: <dbgpt.app.initialization.scheduler.DefaultScheduler object at 0x7f4945bf8310> 2024-06-11 11:24:25 gptai dbgpt.component[15737] INFO Register component with name dbgpt_model_controller and instance: <dbgpt.model.cluster.controller.controller.ModelControllerAdapter object at 0x7f496f7ffa00> 2024-06-11 11:24:25 gptai dbgpt.component[15737] INFO Register component with name dbgpt_connector_manager and instance: <dbgpt.datasource.manages.connector_manager.ConnectorManager object at 0x7f4945bf9390> 2024-06-11 11:24:26 gptai dbgpt.component[15737] INFO Register component with name dbgpt_plugin_hub and instance: <dbgpt.serve.agent.hub.controller.ModulePlugin object at 0x7f4945bfa860> 2024-06-11 11:24:26 gptai dbgpt.component[15737] INFO Register component with name dbgpt_multi_agents and instance: <dbgpt.serve.agent.agents.controller.MultiAgents object at 0x7f4946847400> 2024-06-11 11:24:26 gptai dbgpt.app.initialization.embedding_component[15737] INFO Register local LocalEmbeddingFactory 2024-06-11 11:24:26 gptai dbgpt.model.adapter.embeddings_loader[15737] INFO [EmbeddingsModelWorker] Parameters of device is None, use cuda 2024-06-11 11:24:26 gptai dbgpt.app.initialization.embedding_component[15737] INFO

=========================== EmbeddingModelParameters ===========================

model_name: text2vec model_path: /media/data/xgp/repo/DB-GPT/models/text2vec-large-chinese device: cuda normalize_embeddings: None rerank: False max_length: None

======================================================================

2024-06-11 11:24:28 gptai sentence_transformers.SentenceTransformer[15737] INFO Load pretrained SentenceTransformer: /media/data/xgp/repo/DB-GPT/models/text2vec-large-chinese 2024-06-11 11:24:28 gptai sentence_transformers.SentenceTransformer[15737] WARNING No sentence-transformers model found with name /media/data/xgp/repo/DB-GPT/models/text2vec-large-chinese. Creating a new one with MEAN pooling. 2024-06-11 11:24:56 gptai dbgpt.component[15737] INFO Register component with name embedding_factory and instance: <dbgpt.app.initialization.embedding_component.LocalEmbeddingFactory object at 0x7f4945a3d7e0> 2024-06-11 11:24:57 gptai dbgpt.component[15737] INFO Register component with name dbgpt_model_cache_manager and instance: <dbgpt.storage.cache.manager.LocalCacheManager object at 0x7f4899a80d90> 2024-06-11 11:24:57 gptai dbgpt.component[15737] INFO Register component with name dbgpt_awel_trigger_manager and instance: <dbgpt.core.awel.trigger.trigger_manager.DefaultTriggerManager object at 0x7f4899a80d60> 2024-06-11 11:24:57 gptai dbgpt.component[15737] INFO Register component with name dbgpt_awel_dag_manager and instance: <dbgpt.core.awel.dag.dag_manager.DAGManager object at 0x7f4899a81540> 2024-06-11 11:24:58 gptai dbgpt.component[15737] INFO Register component with name dbgpt_resource_manager and instance: <dbgpt.agent.resource.manage.ResourceManager object at 0x7f4945ce6d10> 2024-06-11 11:24:58 gptai dbgpt.component[15737] INFO Register component with name dbgpt_agent_manager and instance: <dbgpt.agent.core.agent_manage.AgentManager object at 0x7f4899a81690> 2024-06-11 11:24:58 gptai dbgpt.component[15737] INFO Register component with name dbgpt_app_editor_service and instance: <dbgpt.app.openapi.api_v1.editor.service.EditorService object at 0x7f489474c130> 2024-06-11 11:24:58 gptai dbgpt.component[15737] INFO Register component with name dbgpt_serve_prompt and instance: <dbgpt.serve.prompt.serve.Serve object at 0x7f489474c760> 2024-06-11 11:24:58 gptai dbgpt.component[15737] INFO Register component with name dbgpt_serve_prompt_service and instance: <dbgpt.serve.prompt.service.service.Service object at 0x7f489474c070> 2024-06-11 11:24:58 gptai dbgpt.component[15737] INFO Register component with name dbgpt_serve_conversation and instance: <dbgpt.serve.conversation.serve.Serve object at 0x7f4894614d60> 2024-06-11 11:24:58 gptai dbgpt.component[15737] INFO Register component with name dbgpt_serve_conversation_service and instance: <dbgpt.serve.conversation.service.service.Service object at 0x7f4894614df0> 2024-06-11 11:24:58 gptai dbgpt.component[15737] INFO Register component with name dbgpt_serve_flow and instance: <dbgpt.serve.flow.serve.Serve object at 0x7f489457d540> 2024-06-11 11:24:58 gptai dbgpt.component[15737] INFO Register component with name dbgpt_dbgpts_loader and instance: <dbgpt.util.dbgpts.loader.DBGPTsLoader object at 0x7f487ccbaa10> 2024-06-11 11:24:58 gptai dbgpt.component[15737] INFO Register component with name dbgpt_serve_flow_service and instance: <dbgpt.serve.flow.service.service.Service object at 0x7f489457d5d0> 2024-06-11 11:24:58 gptai dbgpt.component[15737] INFO Register component with name dbgpt_rag and instance: <dbgpt.serve.rag.serve.Serve object at 0x7f487cb99210> 2024-06-11 11:24:58 gptai dbgpt.component[15737] INFO Register component with name dbgpt_rag_service and instance: <dbgpt.serve.rag.service.service.Service object at 0x7f487cb99390> 2024-06-11 11:24:59 gptai dbgpt.component[15737] INFO Register component with name dbgpt_datasource and instance: <dbgpt.serve.datasource.serve.Serve object at 0x7f487c20cf70> 2024-06-11 11:24:59 gptai dbgpt.component[15737] INFO Register component with name dbgpt_datasource_service and instance: <dbgpt.serve.datasource.service.service.Service object at 0x7f487c20cd30> 2024-06-11 11:24:59 gptai dbgpt.util._db_migration_utils[15737] WARNING Initialize and upgrade database metadata with alembic, just run this in your development environment, if you deploy this in production environment, please run webserver with --disable_alembic_upgrade(python dbgpt/app/dbgpt_server.py --disable_alembic_upgrade). we suggest you to use dbgpt db migration to initialize and upgrade database metadata with alembic, your can run dbgpt db migration --help to get more information. 2024-06-11 11:24:59 gptai alembic.runtime.migration[15737] INFO Context impl SQLiteImpl. 2024-06-11 11:24:59 gptai alembic.runtime.migration[15737] INFO Will assume non-transactional DDL. 2024-06-11 11:25:00 gptai dbgpt.util._db_migration_utils[15737] INFO Migration versions and their file paths: ========================================Migration versions========================================

35a3ab34a8fc (current): New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/35a3ab34a8fc_new_migration.py) c280de729153 : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/c280de729153_new_migration.py) 56c958f52526 : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/56c958f52526_new_migration.py) 518e3c8d2a73 : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/518e3c8d2a73_new_migration.py) 4751478db6d1 : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/4751478db6d1_new_migration.py) 4cd81e5c7c04 : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/4cd81e5c7c04_new_migration.py) b30396026395 : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/b30396026395_new_migration.py) 1d311466de0d : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/1d311466de0d_new_migration.py) 3a6925353580 : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/3a6925353580_new_migration.py) 2ebbf5265dc1 : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/2ebbf5265dc1_new_migration.py) 815ba15eb004 : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/815ba15eb004_new_migration.py) e97d4c8a4513 : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/e97d4c8a4513_new_migration.py) 3f6f3600e88c : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/3f6f3600e88c_new_migration.py) ac07d6157bbd : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/ac07d6157bbd_new_migration.py) 06e98176dbd4 : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/06e98176dbd4_new_migration.py) a55927e6ae78 : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/a55927e6ae78_new_migration.py) 839d9443cadf : New migration (Path: /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/839d9443cadf_new_migration.py)

2024-06-11 11:25:00 gptai alembic.runtime.migration[15737] INFO Context impl SQLiteImpl. 2024-06-11 11:25:00 gptai alembic.runtime.migration[15737] INFO Will assume non-transactional DDL. 2024-06-11 11:25:00 gptai alembic.runtime.migration[15737] INFO Context impl SQLiteImpl. 2024-06-11 11:25:00 gptai alembic.runtime.migration[15737] INFO Will assume non-transactional DDL. 2024-06-11 11:25:00 gptai dbgpt.util._db_migration_utils[15737] INFO alembic migration current revision: 35a3ab34a8fc, latest revision: 35a3ab34a8fc 2024-06-11 11:25:00 gptai alembic.runtime.migration[15737] INFO Context impl SQLiteImpl. 2024-06-11 11:25:00 gptai alembic.runtime.migration[15737] INFO Will assume non-transactional DDL. Generating /media/data/xgp/repo/DB-GPT/pilot/meta_data/alembic/versions/3fc461ac764c_new_migration.py ... done 2024-06-11 11:25:00 gptai alembic.runtime.migration[15737] INFO Context impl SQLiteImpl. 2024-06-11 11:25:00 gptai alembic.runtime.migration[15737] INFO Will assume non-transactional DDL. 2024-06-11 11:25:00 gptai alembic.runtime.migration[15737] INFO Running upgrade 35a3ab34a8fc -> 3fc461ac764c, New migration model_path:/media/data/llm/Qwen-14B-Chat CFG.LLM_MODEL_PATH:/media/data/llm/Qwen-14B-Chat None Model Unified Deployment Mode! 2024-06-11 11:25:00 gptai dbgpt.model.cluster.worker.manager[15737] INFO Worker params:

**=========================== ModelWorkerParameters ===========================

model_name: text2vec model_path: /media/data/llm/Qwen-14B-Chat worker_type: None model_alias: None worker_class: None model_type: huggingface** host: 10.0.18.15 port: 6006 daemon: False limit_model_concurrency: 5 standalone: True register: True worker_register_host: None controller_addr: None send_heartbeat: True heartbeat_interval: 20 log_level: None log_file: dbgpt_model_worker_manager.log tracer_file: dbgpt_model_worker_manager_tracer.jsonl tracer_storage_cls: None

======================================================================

2024-06-11 11:25:00 gptai dbgpt.model.cluster.worker.manager[15737] INFO Run WorkerManager with standalone mode, controller_addr: http://127.0.0.1:6006 2024-06-11 11:25:00 gptai dbgpt.model.adapter.model_adapter[15737] INFO Use fastcat adapter 2024-06-11 11:25:01 gptai datasets[15737] INFO PyTorch version 2.2.1 available. Warning: Loading base MPT model with zero_shot conversation configuration. If this is not desired, inspect model configurations and names. 2024-06-11 11:25:02 gptai dbgpt.model.adapter.fschat_adapter[15737] INFO Found llm model adapter with model name: text2vec, <fastchat.model.model_adapter.BaseModelAdapter object at 0x7f487691c0a0> 2024-06-11 11:25:02 gptai dbgpt.model.cluster.worker.default_worker[15737] INFO model_name: text2vec, model_path: /media/data/llm/Qwen-14B-Chat, model_param_class: <class 'dbgpt.model.parameter.ModelParameters'> 2024-06-11 11:25:02 gptai dbgpt.model.cluster.worker.default_worker[15737] INFO [DefaultModelWorker] Parameters of device is None, use cuda 2024-06-11 11:25:02 gptai dbgpt.model.cluster.worker.manager[15737] INFO Init empty instances list for text2vec@llm 2024-06-11 11:25:02 gptai dbgpt.component[15737] INFO Register component with name dbgpt_worker_manager_factory and instance: <dbgpt.model.cluster.worker.manager._DefaultWorkerManagerFactory object at 0x7f4876b3dd50> 2024-06-11 11:25:02 gptai dbgpt.util.dbgpts.loader[15737] INFO Found 0 dbgpts packages from /root/.dbgpts/packages/b2ece4820ad656fb1994d4b049587c07 INFO: Started server process [15737] INFO: Waiting for application startup. 2024-06-11 11:25:02 gptai dbgpt.core.awel.dag.loader[15737] INFO Importing /media/data/xgp/repo/DB-GPT/examples/awel/data_analyst_assistant.py 2024-06-11 11:25:02 gptai dbgpt.core.awel.dag.loader[15737] INFO Found dag DAG(dag_id=dbgpt_awel_data_analyst_assistant) from mod <module 'unusual_prefix_103cda6305b276414b188468a911ff111cf680a0_data_analyst_assistant' from '/media/data/xgp/repo/DB-GPT/examples/awel/data_analyst_assistant.py'> and model file /media/data/xgp/repo/DB-GPT/examples/awel/data_analyst_assistant.py 2024-06-11 11:25:02 gptai dbgpt.core.awel.dag.loader[15737] INFO Importing /media/data/xgp/repo/DB-GPT/examples/awel/simple_chat_dag_example.py 2024-06-11 11:25:02 gptai dbgpt.core.awel.dag.loader[15737] INFO Found dag DAG(dag_id=dbgpt_awel_simple_dag_example) from mod <module 'unusual_prefix_0a66292b4ce32512b8c576e90a72eed2f5f6048e_simple_chat_dag_example' from '/media/data/xgp/repo/DB-GPT/examples/awel/simple_chat_dag_example.py'> and model file /media/data/xgp/repo/DB-GPT/examples/awel/simple_chat_dag_example.py 2024-06-11 11:25:02 gptai dbgpt.core.awel.dag.loader[15737] INFO Importing /media/data/xgp/repo/DB-GPT/examples/awel/simple_chat_history_example.py 2024-06-11 11:25:02 gptai dbgpt.core.awel.dag.loader[15737] INFO Found dag DAG(dag_id=dbgpt_awel_simple_chat_history) from mod <module 'unusual_prefix_49ed53caf77bed9469ba892968dad8c08fb5fb9f_simple_chat_history_example' from '/media/data/xgp/repo/DB-GPT/examples/awel/simple_chat_history_example.py'> and model file /media/data/xgp/repo/DB-GPT/examples/awel/simple_chat_history_example.py 2024-06-11 11:25:02 gptai dbgpt.core.awel.dag.loader[15737] INFO Importing /media/data/xgp/repo/DB-GPT/examples/awel/simple_dag_example.py 2024-06-11 11:25:02 gptai dbgpt.core.awel.dag.loader[15737] INFO Found dag DAG(dag_id=simple_dag_example) from mod <module 'unusual_prefix_755fa68fab276d358bbc56b479c8d18ec40d315c_simple_dag_example' from '/media/data/xgp/repo/DB-GPT/examples/awel/simple_dag_example.py'> and model file /media/data/xgp/repo/DB-GPT/examples/awel/simple_dag_example.py 2024-06-11 11:25:02 gptai dbgpt.core.awel.dag.loader[15737] INFO Importing /media/data/xgp/repo/DB-GPT/examples/awel/simple_llm_client_example.py 2024-06-11 11:25:02 gptai dbgpt.core.awel.dag.loader[15737] INFO Found dag DAG(dag_id=dbgpt_awel_simple_llm_client_generate) from mod <module 'unusual_prefix_9a8fcb447e2b98c8b1339a61411d59f5adf58e0b_simple_llm_client_example' from '/media/data/xgp/repo/DB-GPT/examples/awel/simple_llm_client_example.py'> and model file /media/data/xgp/repo/DB-GPT/examples/awel/simple_llm_client_example.py 2024-06-11 11:25:02 gptai dbgpt.core.awel.dag.loader[15737] INFO Found dag DAG(dag_id=dbgpt_awel_simple_llm_client_count_token) from mod <module 'unusual_prefix_9a8fcb447e2b98c8b1339a61411d59f5adf58e0b_simple_llm_client_example' from '/media/data/xgp/repo/DB-GPT/examples/awel/simple_llm_client_example.py'> and model file /media/data/xgp/repo/DB-GPT/examples/awel/simple_llm_client_example.py 2024-06-11 11:25:02 gptai dbgpt.core.awel.dag.loader[15737] INFO Importing /media/data/xgp/repo/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py 2024-06-11 11:25:04 gptai dbgpt.core.awel.dag.loader[15737] INFO Found dag DAG(dag_id=simple_nl_schema_sql_chart_example) from mod <module 'unusual_prefix_3cc959e0e8d98b84c25b15e2d26c32effd1b97fb_simple_nl_schema_sql_chart_example' from '/media/data/xgp/repo/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py'> and model file /media/data/xgp/repo/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py 2024-06-11 11:25:04 gptai dbgpt.core.awel.dag.loader[15737] INFO Importing /media/data/xgp/repo/DB-GPT/examples/awel/simple_rag_rewrite_example.py 2024-06-11 11:25:04 gptai dbgpt.core.awel.dag.loader[15737] INFO Found dag DAG(dag_id=dbgpt_awel_simple_rag_rewrite_example) from mod <module 'unusual_prefix_6edc95ff890adbb8889ce175a0444f37c3c35454_simple_rag_rewrite_example' from '/media/data/xgp/repo/DB-GPT/examples/awel/simple_rag_rewrite_example.py'> and model file /media/data/xgp/repo/DB-GPT/examples/awel/simple_rag_rewrite_example.py 2024-06-11 11:25:04 gptai dbgpt.core.awel.dag.loader[15737] INFO Importing /media/data/xgp/repo/DB-GPT/examples/awel/simple_rag_summary_example.py 2024-06-11 11:25:04 gptai dbgpt.core.awel.dag.loader[15737] INFO Found dag DAG(dag_id=dbgpt_awel_simple_rag_summary_example) from mod <module 'unusual_prefix_bb53c27414ddd14dbf941977f99a9345517d7eb7_simple_rag_summary_example' from '/media/data/xgp/repo/DB-GPT/examples/awel/simple_rag_summary_example.py'> and model file /media/data/xgp/repo/DB-GPT/examples/awel/simple_rag_summary_example.py 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Register trigger HttpTrigger(node_id=ad3e6222-b31c-40cb-a67d-188f191aa00a) 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7f486d685bd0>(AWEL_trigger_routeexamples_data_analyst_copilot), endpoint: /examples/data_analyst/copilot, methods: ['POST'] 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/data_analyst/copilot 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Include router <fastapi.routing.APIRouter object at 0x7f4899a80f40> to prefix path /api/v1/awel/trigger 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Register trigger HttpTrigger(node_id=ae88dac4-70bd-4bac-9641-7b0048e62acb) 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7f486d686200>(AWEL_trigger_routeexamples_simple_chat), endpoint: /examples/simple_chat, methods: ['POST'] 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_chat 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Include router <fastapi.routing.APIRouter object at 0x7f4899a80f40> to prefix path /api/v1/awel/trigger 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Register trigger HttpTrigger(node_id=9da508b1-bec0-4a77-8378-5ec0a9db47ac) 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7f486d687010>(AWEL_trigger_routeexamples_simple_history_multi_round_chat_completions), endpoint: /examples/simple_history/multi_round/chat/completions, methods: ['POST'] 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_history/multi_round/chat/completions 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Include router <fastapi.routing.APIRouter object at 0x7f4899a80f40> to prefix path /api/v1/awel/trigger 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Register trigger HttpTrigger(node_id=68ab48b6-8e28-4bb9-850a-73ffe9101687) 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function_get at 0x7f486d687640>(AWEL_trigger_routeexamples_hello), endpoint: /examples/hello, methods: ['GET'] 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/hello 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Include router <fastapi.routing.APIRouter object at 0x7f4899a80f40> to prefix path /api/v1/awel/trigger 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Register trigger HttpTrigger(node_id=5b74156a-b33d-46bb-a91f-c8f227743e29) 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7f486d6868c0>(AWEL_trigger_routeexamples_simple_client_chat_completions), endpoint: /examples/simple_client/chat/completions, methods: ['POST'] 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_client/chat/completions 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Include router <fastapi.routing.APIRouter object at 0x7f4899a80f40> to prefix path /api/v1/awel/trigger 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Register trigger HttpTrigger(node_id=4cc882a1-844e-4339-ab63-eb3f89572173) 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7f486d708f70>(AWEL_trigger_route__examples_simple_client_count_token), endpoint: /examples/simple_client/count_token, methods: ['POST'] 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_client/count_token 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Include router <fastapi.routing.APIRouter object at 0x7f4899a80f40> to prefix path /api/v1/awel/trigger 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Register trigger HttpTrigger(node_id=94bdc39e-f480-435f-af6a-6bc0915435f8) 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7f486d709cf0>(AWEL_trigger_routeexamples_rag_schema_linking), endpoint: /examples/rag/schema_linking, methods: ['POST'] 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/rag/schema_linking 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Include router <fastapi.routing.APIRouter object at 0x7f4899a80f40> to prefix path /api/v1/awel/trigger 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Register trigger HttpTrigger(node_id=f6cd4a8e-a144-49a0-8c59-dc14774576f4) 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7f486d6860e0>(AWEL_trigger_route__examples_rag_rewrite), endpoint: /examples/rag/rewrite, methods: ['POST'] 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/rag/rewrite 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Include router <fastapi.routing.APIRouter object at 0x7f4899a80f40> to prefix path /api/v1/awel/trigger 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Register trigger HttpTrigger(node_id=f39a741b-2323-4c02-b632-e7ea6188796b) 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7f486d70b010>(AWEL_trigger_route__examples_rag_summary), endpoint: /examples/rag/summary, methods: ['POST'] 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.http_trigger[15737] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/rag/summary 2024-06-11 11:25:04 gptai dbgpt.core.awel.trigger.trigger_manager[15737] INFO Include router <fastapi.routing.APIRouter object at 0x7f4899a80f40> to prefix path /api/v1/awel/trigger 2024-06-11 11:25:05 gptai dbgpt.model.cluster.worker.manager[15737] INFO Begin start all worker, apply_req: None 2024-06-11 11:25:05 gptai dbgpt.model.cluster.worker.manager[15737] INFO Apply req: None, apply_func: <function LocalWorkerManager._start_all_worker.._start_worker at 0x7f486d483880> 2024-06-11 11:25:05 gptai dbgpt.model.cluster.worker.manager[15737] INFO Apply to all workers INFO: Application startup complete. 2024-06-11 11:25:05 gptai dbgpt.model.cluster.worker.default_worker[15737] INFO Begin load model, model params:

=========================== ModelParameters ===========================

model_name: text2vec model_path: /media/data/llm/Qwen-14B-Chat device: cuda model_type: huggingface prompt_template: None max_context_size: 4096 num_gpus: None max_gpu_memory: None cpu_offloading: False load_8bit: True load_4bit: False quant_type: nf4 use_double_quant: True compute_dtype: None trust_remote_code: True verbose: False

======================================================================

INFO: Uvicorn running on http://10.0.18.15:6006 (Press CTRL+C to quit) 2024-06-11 11:25:05 gptai dbgpt.model.adapter.loader[15737] WARNING Current model text2vec not supported quantization 2024-06-11 11:25:07 gptai transformers_modules.Qwen-14B-Chat.modeling_qwen[15737] WARNING The model is automatically converting to bf16 for faster inference. If you want to disable the automatic precision, please manually add bf16/fp16/fp32=True to "AutoModelForCausalLM.from_pretrained". 2024-06-11 11:25:07 gptai transformers_modules.Qwen-14B-Chat.modeling_qwen[15737] WARNING Try importing flash-attention for faster inference... 2024-06-11 11:25:07 gptai transformers_modules.Qwen-14B-Chat.modeling_qwen[15737] WARNING Warning: import flash_attn rotary fail, please install FlashAttention rotary to get higher efficiency https://github.com/Dao-AILab/flash-attention/tree/main/csrc/rotary 2024-06-11 11:25:07 gptai transformers_modules.Qwen-14B-Chat.modeling_qwen[15737] WARNING Warning: import flash_attn rms_norm fail, please install FlashAttention layer_norm to get higher efficiency https://github.com/Dao-AILab/flash-attention/tree/main/csrc/layer_norm 2024-06-11 11:25:07 gptai transformers_modules.Qwen-14B-Chat.modeling_qwen[15737] WARNING Warning: import flash_attn fail, please install FlashAttention to get higher efficiency https://github.com/Dao-AILab/flash-attention Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [03:52<00:00, 15.51s/it] 2024-06-11 11:29:00 gptai dbgpt.model.cluster.worker.default_worker[15737] INFO Parse model max length 8192 from model text2vec. 2024-06-11 11:29:00 gptai dbgpt.model.cluster.worker.manager[15737] ERROR Error starting worker manager: model text2vec@huggingface(10.0.18.15:6006) start failed for network error, please make sure your port is available, if you are using global network proxy, please close it INFO: Shutting down INFO: Waiting for application shutdown. 2024-06-11 11:29:00 gptai dbgpt.model.cluster.worker.manager[15737] INFO Stop all workers 2024-06-11 11:29:00 gptai dbgpt.model.cluster.worker.manager[15737] INFO Apply req: None, apply_func: <function LocalWorkerManager._stop_all_worker.._stop_worker at 0x7f486d483880> 2024-06-11 11:29:00 gptai dbgpt.model.cluster.worker.manager[15737] INFO Apply to all workers 2024-06-11 11:29:01 gptai dbgpt.model.cluster.worker.manager[15737] WARNING Stop worker, ignored exception from deregister_func: All connection attempts failed /media/data/xgp/repo/DB-GPT/dbgpt/util/model_utils.py:26: UserWarning: 'has_mps' is deprecated, please use 'torch.backends.mps.is_built()' if torch.has_mps: /media/data/xgp/repo/DB-GPT/dbgpt/util/model_utils.py:33: UserWarning: 'has_cuda' is deprecated, please use 'torch.backends.cuda.is_built()' elif torch.has_cuda: 2024-06-11 11:29:01 gptai dbgpt.util.model_utils[15737] INFO Clear torch cache of device: cuda:0 2024-06-11 11:29:01 gptai dbgpt.util.model_utils[15737] INFO Clear torch cache of device: cuda:1 2024-06-11 11:29:01 gptai dbgpt.model.cluster.worker.manager[15737] WARNING Stop worker, ignored exception from deregister_func: All connection attempts failed INFO: Application shutdown complete. INFO: Finished server process [15737]

What happened

INFO: Uvicorn running on http://0.0.0.0:6006 (Press CTRL+C to quit) 2024-06-07 18:47:50 gptai dbgpt.model.adapter.loader[3506271] WARNING Current model /media/data/xgp/repo/DB-GPT/models/text2vec-large-chinese not supported quantization If you want to use BertLMHeadModel as a standalone, add is_decoder=True. 2024-06-07 18:47:50 gptai dbgpt.model.cluster.worker.manager[3506271] ERROR Error starting worker manager: model /media/data/xgp/repo/DB-GPT/models/text2vec-large-chinese@huggingface(10.0.18.15:6006) start failed, Traceback (most recent call last): File "/media/data/xgp/repo/DB-GPT/dbgpt/model/cluster/worker/manager.py", line 507, in _start_worker await self.run_blocking_func( File "/media/data/xgp/repo/DB-GPT/dbgpt/model/cluster/worker/manager.py", line 105, in run_blocking_func return await loop.run_in_executor(self.executor, func, args) File "/root/anaconda3/envs/dbgpt_env/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(self.args, self.kwargs) File "/media/data/xgp/repo/DB-GPT/dbgpt/model/cluster/worker/default_worker.py", line 116, in start self.model, self.tokenizer = self.ml.loader_with_params( File "/media/data/xgp/repo/DB-GPT/dbgpt/model/adapter/loader.py", line 125, in loader_with_params return huggingface_loader(llm_adapter, model_params) File "/media/data/xgp/repo/DB-GPT/dbgpt/model/adapter/loader.py", line 207, in huggingface_loader model, tokenizer = llm_adapter.load(model_params.model_path, kwargs) File "/media/data/xgp/repo/DB-GPT/dbgpt/model/adapter/fschat_adapter.py", line 112, in load return self._adapter.load_model(model_path, from_pretrained_kwargs) File "/root/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/fastchat/model/model_adapter.py", line 99, in load_model model = AutoModelForCausalLM.from_pretrained( File "/root/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 563, in from_pretrained return model_class.from_pretrained( File "/root/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3669, in from_pretrained no_split_modules = model._get_no_split_modules(device_map) File "/root/anaconda3/envs/dbgpt_env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 1892, in _get_no_split_modules raise ValueError( ValueError: BertLMHeadModel does not support device_map='auto'. To implement support, the model class needs to implement the _no_split_modules attribute.**

What you expected to happen

可以正常运行一个带embedding和大语言模型的demo

How to reproduce

export LLM_MODEL_PATH=/media/data/llm/Qwen-14B-Chat python ./dbgpt/app/dbgpt_server.py --host 10.0.18.15 --port 6006

Additional context

No response

Are you willing to submit PR?

fangyinc commented 3 months ago

Try run DB-GPT with following command:

python ./dbgpt/app/dbgpt_server.py  --port 6006
chuangzhidan commented 3 months ago

Try run DB-GPT with following command:

python ./dbgpt/app/dbgpt_server.py  --port 6006

thank you so much !
can i kindld ask ,what's the control addr for ? do you know where i set import params?, say proxy,in file .env? didn't seem to work before. looked up in https://www.yuque.com/ too,its has'nt been updated for a long time. didn't find the answer. so far ,i did in linux environment,"export LLM_MODEL_PATH=" and such.

fangyinc commented 3 months ago

Try run DB-GPT with following command:

python ./dbgpt/app/dbgpt_server.py  --port 6006

thank you so much ! can i kindld ask ,what's the control addr for ? do you know where i set import params?, say proxy,in file .env? didn't seem to work before. looked up in https://www.yuque.com/ too,its has'nt been updated for a long time. didn't find the answer. so far ,i did in linux environment,"export LLM_MODEL_PATH=" and such.

The controller_addr is used to connect to the model cluster service started by DB-GPT. On most systems, environment variables in .env files can be read normally.

chuangzhidan commented 3 months ago

Try run DB-GPT with following command:

python ./dbgpt/app/dbgpt_server.py  --port 6006

thank you so much ! can i kindld ask ,what's the control addr for ? do you know where i set import params?, say proxy,in file .env? didn't seem to work before. looked up in https://www.yuque.com/ too,its has'nt been updated for a long time. didn't find the answer. so far ,i did in linux environment,"export LLM_MODEL_PATH=" and such.

The controller_addr is used to connect to the model cluster service started by DB-GPT. On most systems, environment variables in .env files can be read normally.

❥(^_-)