search

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, GraphRAG, DeepSpeed, Axolotl, etc
Apache License 2.0
6.69k stars 1.26k forks source link

Install ipex-llm is resulting in timeout #10551

Open shailesh837 opened 7 months ago

shailesh837 commented 7 months ago

I am following new guide for installing ipex on Intel Arc 770 GPU: I added : pip install --default-timeout=100 future , But now its horrible slow https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_linux_gpu.html Install ipex-llm:

`(llm) serpent@serpent-NUC12SNKi72:~/LLM_Demo$ pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://developer.intel.com/ipex-whl-stable-xpu Looking in indexes: https://pypi.org/simple, https://developer.intel.com/ipex-whl-stable-xpu Collecting ipex-llm[xpu] Using cached ipex_llm-2.1.0b20240326-py3-none-manylinux2010_x86_64.whl.metadata (4.5 kB) Collecting py-cpuinfo (from ipex-llm[xpu]) Using cached py_cpuinfo-9.0.0-py3-none-any.whl.metadata (794 bytes) Collecting protobuf (from ipex-llm[xpu]) Using cached protobuf-5.26.0-cp37-abi3-manylinux2014_x86_64.whl.metadata (592 bytes) Collecting mpmath==1.3.0 (from ipex-llm[xpu]) Using cached mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB) Collecting numpy==1.26.4 (from ipex-llm[xpu]) Using cached numpy-1.26.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB) Collecting transformers==4.31.0 (from ipex-llm[xpu]) Using cached transformers-4.31.0-py3-none-any.whl.metadata (116 kB) Collecting sentencepiece (from ipex-llm[xpu]) Using cached sentencepiece-0.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.7 kB) Collecting tokenizers==0.13.3 (from ipex-llm[xpu]) Using cached tokenizers-0.13.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB) Collecting accelerate==0.21.0 (from ipex-llm[xpu]) Using cached accelerate-0.21.0-py3-none-any.whl.metadata (17 kB) Collecting tabulate (from ipex-llm[xpu]) Using cached tabulate-0.9.0-py3-none-any.whl.metadata (34 kB) Collecting torch==2.1.0a0 (from ipex-llm[xpu]) Downloading https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/./torch-2.1.0a0%2Bcxx11.abi-cp39-cp39-linux_x86_64.whl (150.8 MB) ╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/150.8 MB 48.0 kB/s eta 0:51:19 ╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/150.8 MB 48.0 kB/s eta 0:51:19 ERROR: Exception: Traceback (most recent call last): File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/urllib3/response.py", line 438, in _error_catcher yield File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/urllib3/response.py", line 561, in read data = self._fp_read(amt) if not fp_closed else b"" File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/urllib3/response.py", line 527, in _fp_read return self._fp.read(amt) if amt is not None else self._fp.read() File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 98, in read data: bytes = self.__fp.read(amt) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/http/client.py", line 463, in read n = self.readinto(b) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/http/client.py", line 507, in readinto n = self.fp.readinto(b) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/socket.py", line 704, in readinto return self._sock.recv_into(b) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/ssl.py", line 1275, in recv_into return self.read(nbytes, buffer) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/ssl.py", line 1133, in read return self._sslobj.read(len, buffer) socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper status = run_func(*args) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper return func(self, options, args) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 377, in run requirement_set = resolver.resolve( File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve result = self._result = resolver.resolve( File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve state = resolution.resolve(requirements, max_rounds=max_rounds) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 427, in resolve failure_causes = self._attempt_to_pin_criterion(name) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 239, in _attempt_to_pin_criterion criteria = self._get_updated_criteria(candidate) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 230, in _get_updated_criteria self._add_to_criteria(criteria, requirement, parent=candidate) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria if not criterion.candidates: File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in bool return bool(self._sequence) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in bool return any(self) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in return (c for c in iterator if id(c) not in self._incompatible_ids) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built candidate = func() File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 211, in _make_candidate_from_link self._link_candidate_cache[link] = LinkCandidate( File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 293, in init super().init( File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in init self.dist = self._prepare() File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare dist = self._prepare_distribution() File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 304, in _prepare_distribution return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 525, in prepare_linked_requirement return self._prepare_linked_requirement(req, parallel_builds) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 596, in _prepare_linked_requirement local_file = unpack_url( File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 168, in unpack_url file = get_http_url( File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 109, in get_http_url from_path, content_type = download(link, temp_dir.path) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/network/download.py", line 147, in call for chunk in chunks: File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/cli/progress_bars.py", line 53, in _rich_progress_bar for chunk in iterable: File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_internal/network/utils.py", line 63, in response_chunks for chunk in response.raw.stream( File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/urllib3/response.py", line 622, in stream data = self.read(amt=amt, decode_content=decode_content) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/urllib3/response.py", line 587, in read raise IncompleteRead(self._fp_bytes_read, self.length_remaining) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/contextlib.py", line 137, in exit self.gen.throw(typ, value, traceback) File "/home/serpent/miniconda3/envs/llm/lib/python3.9/site-packages/pip/_vendor/urllib3/response.py", line 443, in _error_catcher raise ReadTimeoutError(self._pool, None, "Read timed out.") pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='intel-extension-for-pytorch.s3.amazonaws.com', port=443): Read timed out. `

hkvision commented 7 months ago

Hi, we have updated our installation command in our documents, sorry for the confusion. Check this page: https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Overview/install_gpu.html#id3 Try pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/ instead :)

Feel free to tell us if the issue still exists.