NoneJou072 / robochain

A simulation framework based on ROS2 and LLMs(like GPT) for robot interaction tasks in the era of large models
MIT License
111 stars 12 forks source link

我有 openai api 不需要huggingface的模型,我在配置的setup.py source 后,为什么还需要我huggingface llama #20

Open wisepmlin opened 6 months ago

wisepmlin commented 6 months ago
截屏2024-05-21 20 30 43

yahboom@yahboom-virtual-machine:~/gpt_ws$ ros2 run gpt_client gpt_client [INFO] [1716294495.241716529] [gpt_client]: gpt_client already. INFO:root:Loading keys... INFO:root:Done. INFO:root:Initialize LLM... INFO:root:Done. INFO:root:Initialize tools... /usr/lib/python3/dist-packages/scipy/init.py:146: UserWarning: A NumPy version >=1.17.3 and <1.25.0 is required for this version of SciPy (detected version 1.26.4 warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}" INFO:sentence_transformers.SentenceTransformer:Load pretrained SentenceTransformer: flax-sentence-embeddings/all_datasets_v4_MiniLM-L6 WARNING:sentence_transformers.SentenceTransformer:No sentence-transformers model found with name flax-sentence-embeddings/all_datasets_v4_MiniLM-L6. Creating a new one with MEAN pooling. /home/yahboom/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: resume_download is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use force_download=True. warnings.warn( Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 169, in _new_conn conn = connection.create_connection( File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 96, in create_connection raise err File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 86, in create_connection sock.connect(sa) TimeoutError: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 700, in urlopen httplib_response = self._make_request( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 383, in _make_request self._validate_conn(conn) File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 1017, in _validate_conn conn.connect() File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 353, in connect conn = self._new_conn() File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 174, in _new_conn raise ConnectTimeoutError( urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x79af20146fe0>, 'Connection to huggingface.co timed out. (connect timeout=10)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yahboom/.local/lib/python3.10/site-packages/requests/adapters.py", line 564, in send resp = conn.urlopen( File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 756, in urlopen retries = retries.increment( File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 574, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /flax-sentence-embeddings/all_datasets_v4_MiniLM-L6/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x79af20146fe0>, 'Connection to huggingface.co timed out. (connect timeout=10)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yahboom/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1722, in _get_metadata_or_catch_error metadata = get_hf_file_metadata(url=url, proxies=proxies, timeout=etag_timeout, headers=headers) File "/home/yahboom/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(args, kwargs) File "/home/yahboom/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1645, in get_hf_file_metadata r = _request_wrapper( File "/home/yahboom/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 372, in _request_wrapper response = _request_wrapper( File "/home/yahboom/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 395, in _request_wrapper response = get_session().request(method=method, url=url, params) File "/home/yahboom/.local/lib/python3.10/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) File "/home/yahboom/.local/lib/python3.10/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, kwargs) File "/home/yahboom/.local/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 66, in send return super().send(request, args, **kwargs) File "/home/yahboom/.local/lib/python3.10/site-packages/requests/adapters.py", line 585, in send raise ConnectTimeout(e, request=request) requests.exceptions.ConnectTimeout: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /flax-sentence-embeddings/all_datasets_v4_MiniLM-L6/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x79af20146fe0>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 0f6e6172-036d-4e0d-b16e-a19abe9f47a4)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/yahboom/.local/lib/python3.10/site-packages/transformers/utils/hub.py", line 399, in cached_file resolved_file = hf_hub_download( File "/home/yahboom/.local/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn return fn(*args, **kwargs) File "/home/yahboom/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download return _hf_hub_download_to_cache_dir( File "/home/yahboom/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1325, in _hf_hub_download_to_cache_dir _raise_on_head_call_error(head_call_error, force_download, local_files_only) File "/home/yahboom/.local/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1826, in _raise_on_head_call_error raise LocalEntryNotFoundError( huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/yahboom/gpt_ws/install/gpt_client/lib/gpt_client/gpt_client", line 33, in sys.exit(load_entry_point('gpt-client==0.0.0', 'console_scripts', 'gpt_client')()) File "/home/yahboom/gpt_ws/install/gpt_client/lib/python3.10/site-packages/gpt_client/examples/client_retrieval_gpt.py", line 103, in main gpt_node = GPTClient( File "/home/yahboom/gpt_ws/install/gpt_client/lib/python3.10/site-packages/gpt_client/examples/client_retrieval_gpt.py", line 81, in init self.gpt = GPTAssistant( File "/home/yahboom/gpt_ws/install/gpt_client/lib/python3.10/site-packages/gpt_client/examples/client_retrieval_gpt.py", line 39, in init embedding_model = eu.init_embedding_model() File "/home/yahboom/gpt_ws/install/gpt_client/lib/python3.10/site-packages/gpt_client/commons/embedding_utils.py", line 39, in init_embedding_model embeddings = HuggingFaceEmbeddings( File "/home/yahboom/.local/lib/python3.10/site-packages/langchain_community/embeddings/huggingface.py", line 72, in init self.client = sentence_transformers.SentenceTransformer( File "/home/yahboom/.local/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 205, in init modules = self._load_auto_model( File "/home/yahboom/.local/lib/python3.10/site-packages/sentence_transformers/SentenceTransformer.py", line 1197, in _load_auto_model transformer_model = Transformer( File "/home/yahboom/.local/lib/python3.10/site-packages/sentence_transformers/models/Transformer.py", line 35, in init config = AutoConfig.from_pretrained(model_name_or_path, model_args, cache_dir=cache_dir) File "/home/yahboom/.local/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 934, in from_pretrained config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, kwargs) File "/home/yahboom/.local/lib/python3.10/site-packages/transformers/configuration_utils.py", line 632, in get_config_dict config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs) File "/home/yahboom/.local/lib/python3.10/site-packages/transformers/configuration_utils.py", line 689, in _get_config_dict resolved_config_file = cached_file( File "/home/yahboom/.local/lib/python3.10/site-packages/transformers/utils/hub.py", line 442, in cached_file raise EnvironmentError( OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like flax-sentence-embeddings/all_datasets_v4_MiniLM-L6 is not the path to a directory containing a file named config.json. Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'. [ros2run]: Process exited with failure 1

wisepmlin commented 6 months ago

RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx [ros2run]: Process exited with failure 1

wisepmlin commented 6 months ago

我可以不实用因为大显卡吗?我是 2018 年的苹果电脑

NoneJou072 commented 6 months ago

你好,根据报错信息来看是在下载 all_datasets_v4_MiniLM-L6 模型时出现了网络问题,这个模型是用来进行 embedding 处理的,并不是 llama。

NoneJou072 commented 6 months ago

您可以手动把 retrival 部分从代码中删掉

wisepmlin commented 6 months ago

我将 cuda 改为 cpu,进一步启动了程序,但是新的问题有出现了 elif EMBEDDING_MODEL == 1: embeddings = HuggingFaceEmbeddings( model_name=model_name, model_kwargs={"device": "cpu"}, ) 终端打印: I am ready to help you with your questions and commands. User💬> INFO:backoff:Backing off send_request(...) for 0.9s (requests.exceptions.SSLError: HTTPSConnectionPool(host='us-api.i.posthog.com', port=443): Max retries exceeded with url: /batch/ (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1007)')))) ERROR:backoff:Giving up send_request(...) after 4 tries (requests.exceptions.SSLError: HTTPSConnectionPool(host='us-api.i.posthog.com', port=443): Max retries exceeded with url: /batch/ (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1007)')))) 我在尝试找到原因,进行重新编译

wisepmlin commented 6 months ago

我成功编译了代码,进入了 user chat 环节

wisepmlin commented 6 months ago
截屏2024-05-22 11 09 25 (2)

我不知道为什么,助理返回了 Assistant🤖> ```python object_pos, object_quat = pri.get_obj_pose("red_block") target_pos = object_pos + np.array([0.0, 0.0, 0.1]) # 10cm above the red block pri.grab("red_block") pri.move(target_pos, object_quat) pri.gripper_ctrl('open') pri.res 但是服务器那边的机械没有动起来

wisepmlin commented 6 months ago

yahboom@yahboom-virtual-machine:~/gpt_ws$ ros2 run gpt_server gpt_server INFO:root:Initializing ROS... [INFO] [1716347886.546544101] [gpt_server]: Gpt server has init. INFO:root:Done. INFO:root:Initializing Simulator... INFO:root:Activated agents: ['arm0'] /home/yahboom/.local/lib/python3.10/site-packages/glfw/init.py:914: GLFWError: (65544) b'Wayland: Window position retrieval not supported' warnings.warn(message, GLFWError) INFO:root:Done. 错误信息,是否配置不准确造成运行时候,出现 error

NoneJou072 commented 6 months ago

截屏2024-05-22 11 09 25 (2) 我不知道为什么,助理返回了 Assistant🤖> ```python object_pos, object_quat = pri.get_obj_pose("red_block") target_pos = object_pos + np.array([0.0, 0.0, 0.1]) # 10cm above the red block pri.grab("red_block") pri.move(target_pos, object_quat) pri.gripper_ctrl('open') pri.res 但是服务器那边的机械没有动起来

是否开启通信了呢?需要将程序中的 is_debug=True 改成 False

wisepmlin commented 6 months ago

ok!可以了

Hongdn commented 3 months ago

截屏2024-05-22 11 09 25 (2) 我不知道为什么,助理返回了 Assistant🤖> ```python object_pos, object_quat = pri.get_obj_pose("red_block") target_pos = object_pos + np.array([0.0, 0.0, 0.1]) # 10cm above the red block pri.grab("red_block") pri.move(target_pos, object_quat) pri.gripper_ctrl('open') pri.res 但是服务器那边的机械没有动起来

是否开启通信了呢?需要将程序中的 is_debug=True 改成 False

请问为什么我第一个命令pick都能正常执行,但第二个指令就无法传输到server端了 catch

Hongdn commented 3 months ago

我看了下pri.move指令似乎不能生效

Hongdn commented 3 months ago

move 这只会抓取回位 之后就不动了