Can't download models anymore, not sure why. Used to work perfectly

[BharatBlade]

I got this warning and I'm having trouble understanding what to do. I recently reset my windows 11 pc running an i7-12700, 3060 ti, and 32gb ddr5 ram, and reinstalled python 3.11.7, Visual studio (including build tools), and CUDA toolkit 12.2

This happens after I run "chatdocs download"

C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\embeddings__init__.py:29: LangChainDeprecationWarning: Importing embeddings from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.embeddings import HuggingFaceInstructEmbeddings.

To install langchain-community run pip install -U langchain-community. warnings.warn( C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\embeddings__init__.py:29: LangChainDeprecationWarning: Importing embeddings from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.embeddings import HuggingFaceEmbeddings.

To install langchain-community run pip install -U langchain-community. warnings.warn( C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\llms__init__.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.llms import CTransformers.

To install langchain-community run pip install -U langchain-community. warnings.warn( C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\llms__init__.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.llms import HuggingFacePipeline.

To install langchain-community run pip install -U langchain-community. warnings.warn(

The traceback is really long, but it ends with this: ConnectionError: (ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)), '(Request ID: 31c0ce51-239b-4d01-83db-b2fb46516f05)')

Any help or advice is greatly appreciated. I've had this project working in the past with amazing results and I'm extremely grateful for the work put into this and it's open source nature. I'm hoping to continue using it as I have in the past. I'm also open to work arounds such as downloading the models manually from huggingface, but would also appreciate being told where I should put them.

[BharatBlade]

So far, I've determined that this doesn't seem to be an issue with chatdocs, as I've seen it in similar projects (I forget if I saw the exact same error in PrivateGPT or LocalGPT, I think it was Local). I'm pretty sure this is specifically an issue with LangChain, but I don't understand the error well enough to properly report it. Hopefully someone else can figure it out.

So far I do have a "working" (still some warnings with LangChain) build on my less powerful laptop, solely because I haven't installed any other user made python programs on it. I'm just gonna not touch it on my laptop until the next update of Langchain, PrivateGPT, and/or ChatDocs brings stability back.

I've tried pip freeze > requirements.txt on my laptop, then pip install -r requirements.txt to my desktop, but that also didn't work. I was getting an error that some of the packages required python < 3.11, even though I only installed 3.11.7 on my laptop (only other version I've ever installed on my laptop was 3.11.5). For reference I almost never code in Python, only Java, so there's a lot about python I really don't know about. I've pretty much given up on trying to make it work on my desktop right now since I have it working on my laptop. Fortunately small language models have become a lot more powerful.

Thanks again for the work you've given over the last year.

[lStarHackerI]

Yes, I also met same problem.

[BharatBlade]

Just to give an update, this is still the traceback after running chatdocs download

please let me know if there's a better way to format this from the terminal

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:7 │ │ 90 in urlopen │ │ │ │ 787 │ │ │ response_conn = conn if not release_conn else None │ │ 788 │ │ │ │ │ 789 │ │ │ # Make the request on the HTTPConnection object │ │ ❱ 790 │ │ │ response = self._make_request( │ │ 791 │ │ │ │ conn, │ │ 792 │ │ │ │ method, │ │ 793 │ │ │ │ url, │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ assert_same_host = False │ │ │ │ body = None │ │ │ │ body_pos = None │ │ │ │ chunked = False │ │ │ │ clean_exit = False │ │ │ │ conn = None │ │ │ │ decode_content = False │ │ │ │ destination_scheme = None │ │ │ │ err = None │ │ │ │ headers = {'user-agent': 'unknown/None; hf_hub/0.17.3; python/3.11.7; │ │ │ │ torch/2.1.2+cu121', 'Accept-Encoding': 'gzip, deflate, zstd', │ │ │ │ 'Accept': '/', 'Connection': 'keep-alive', 'authorization': 'Bearer │ │ │ │ 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', 'X-Amzn-Trace-Id': │ │ │ │ '603c5e5a-181a-4746-bcd4-3884aa53d505'} │ │ │ │ http_tunnel_required = False │ │ │ │ method = 'GET' │ │ │ │ new_e = ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An │ │ │ │ existing connection was forcibly closed by the remote host', None, │ │ │ │ 10054, None)) │ │ │ │ parsed_url = Url( │ │ │ │ │ scheme=None, │ │ │ │ │ auth=None, │ │ │ │ │ host=None, │ │ │ │ │ port=None, │ │ │ │ │ path='/api/models/hkunlp/instructor-xl', │ │ │ │ │ query=None, │ │ │ │ │ fragment=None │ │ │ │ ) │ │ │ │ pool_timeout = None │ │ │ │ preload_content = False │ │ │ │ redirect = False │ │ │ │ release_conn = False │ │ │ │ release_this_conn = True │ │ │ │ response_conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ response_kw = {} │ │ │ │ retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) │ │ │ │ self = <urllib3.connectionpool.HTTPSConnectionPool object at │ │ │ │ 0x000001E293406E90> │ │ │ │ timeout = Timeout(connect=None, read=None, total=None) │ │ │ │ timeout_obj = Timeout(connect=None, read=None, total=None) │ │ │ │ url = '/api/models/hkunlp/instructor-xl' │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:4 │ │ 91 in _make_request │ │ │ │ 488 │ │ │ │ new_e, (OSError, NewConnectionError, TimeoutError, SSLError) │ │ 489 │ │ │ ) and (conn and conn.proxy and not conn.has_connected_to_proxy): │ │ 490 │ │ │ │ new_e = _wrap_proxy_error(new_e, conn.proxy.scheme) │ │ ❱ 491 │ │ │ raise new_e │ │ 492 │ │ │ │ 493 │ │ # conn.request() calls http.client..request, not the method in │ │ 494 │ │ # urllib3.request. It also calls makefile (recv) on the socket. │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ body = None │ │ │ │ chunked = False │ │ │ │ conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ decode_content = False │ │ │ │ enforce_content_length = True │ │ │ │ headers = {'user-agent': 'unknown/None; hf_hub/0.17.3; python/3.11.7; │ │ │ │ torch/2.1.2+cu121', 'Accept-Encoding': 'gzip, deflate, zstd', │ │ │ │ 'Accept': '/', 'Connection': 'keep-alive', 'authorization': │ │ │ │ 'Bearer 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', 'X-Amzn-Trace-Id': │ │ │ │ '603c5e5a-181a-4746-bcd4-3884aa53d505'} │ │ │ │ method = 'GET' │ │ │ │ new_e = ConnectionResetError(10054, 'An existing connection was forcibly │ │ │ │ closed by the remote host', None, 10054, None) │ │ │ │ preload_content = False │ │ │ │ response_conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ retries = Retry(total=0, connect=None, read=False, redirect=None, │ │ │ │ status=None) │ │ │ │ self = <urllib3.connectionpool.HTTPSConnectionPool object at │ │ │ │ 0x000001E293406E90> │ │ │ │ timeout = Timeout(connect=None, read=None, total=None) │ │ │ │ timeout_obj = Timeout(connect=None, read=None, total=None) │ │ │ │ url = '/api/models/hkunlp/instructor-xl' │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:4 │ │ 67 in _make_request │ │ │ │ 464 │ │ try: │ │ 465 │ │ │ # Trigger any extra validation we need to do. │ │ 466 │ │ │ try: │ │ ❱ 467 │ │ │ │ self._validate_conn(conn) │ │ 468 │ │ │ except (SocketTimeout, BaseSSLError) as e: │ │ 469 │ │ │ │ self._raise_timeout(err=e, url=url, timeout_value=conn.timeout) │ │ 470 │ │ │ │ raise │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ body = None │ │ │ │ chunked = False │ │ │ │ conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ decode_content = False │ │ │ │ enforce_content_length = True │ │ │ │ headers = {'user-agent': 'unknown/None; hf_hub/0.17.3; python/3.11.7; │ │ │ │ torch/2.1.2+cu121', 'Accept-Encoding': 'gzip, deflate, zstd', │ │ │ │ 'Accept': '/*', 'Connection': 'keep-alive', 'authorization': │ │ │ │ 'Bearer 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', 'X-Amzn-Trace-Id': │ │ │ │ '603c5e5a-181a-4746-bcd4-3884aa53d505'} │ │ │ │ method = 'GET' │ │ │ │ new_e = ConnectionResetError(10054, 'An existing connection was forcibly │ │ │ │ closed by the remote host', None, 10054, None) │ │ │ │ preload_content = False │ │ │ │ response_conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ retries = Retry(total=0, connect=None, read=False, redirect=None, │ │ │ │ status=None) │ │ │ │ self = <urllib3.connectionpool.HTTPSConnectionPool object at │ │ │ │ 0x000001E293406E90> │ │ │ │ timeout = Timeout(connect=None, read=None, total=None) │ │ │ │ timeout_obj = Timeout(connect=None, read=None, total=None) │ │ │ │ url = '/api/models/hkunlp/instructor-xl' │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:1 │ │ 092 in _validate_conn │ │ │ │ 1089 │ │ │ │ 1090 │ │ # Force connect early to allow us to validate the connection. │ │ 1091 │ │ if conn.is_closed: │ │ ❱ 1092 │ │ │ conn.connect() │ │ 1093 │ │ │ │ 1094 │ │ if not conn.is_verified: │ │ 1095 │ │ │ warnings.warn( │ │ │ │ ╭─────────────────────────────────────── locals ────────────────────────────────────────╮ │ │ │ class = <class 'urllib3.connectionpool.HTTPSConnectionPool'> │ │ │ │ conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x000001E293406E90> │ │ │ ╰───────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py:642 │ │ in connect │ │ │ │ 639 │ │ │ │ SystemTimeWarning, │ │ 640 │ │ │ ) │ │ 641 │ │ │ │ ❱ 642 │ │ sock_and_verified = _ssl_wrap_socket_and_match_hostname( │ │ 643 │ │ │ sock=sock, │ │ 644 │ │ │ cert_reqs=self.cert_reqs, │ │ 645 │ │ │ ssl_version=self.ssl_version, │ │ │ │ ╭────────────────────────────────────── locals ───────────────────────────────────────╮ │ │ │ is_time_off = False │ │ │ │ self = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ server_hostname = 'huggingface.co' │ │ │ │ sock = <socket.socket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ tls_in_tls = False │ │ │ ╰─────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py:783 │ │ in _ssl_wrap_socket_and_match_hostname │ │ │ │ 780 │ │ if is_ipaddress(normalized): │ │ 781 │ │ │ server_hostname = normalized │ │ 782 │ │ │ ❱ 783 │ ssl_sock = ssl_wrap_socket( │ │ 784 │ │ sock=sock, │ │ 785 │ │ keyfile=key_file, │ │ 786 │ │ certfile=cert_file, │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ assert_fingerprint = None │ │ │ │ assert_hostname = None │ │ │ │ ca_cert_data = None │ │ │ │ ca_cert_dir = None │ │ │ │ ca_certs = 'C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site… │ │ │ │ cert_file = None │ │ │ │ cert_reqs = 'CERT_REQUIRED' │ │ │ │ context = <ssl.SSLContext object at 0x000001E2958EB020> │ │ │ │ default_ssl_context = True │ │ │ │ key_file = None │ │ │ │ key_password = None │ │ │ │ normalized = 'huggingface.co' │ │ │ │ server_hostname = 'huggingface.co' │ │ │ │ sock = <socket.socket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ ssl_context = None │ │ │ │ ssl_maximum_version = None │ │ │ │ ssl_minimum_version = None │ │ │ │ ssl_version = None │ │ │ │ tls_intls = False │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\ssl.py:469 in │ │ ssl_wrap_socket │ │ │ │ 466 │ except NotImplementedError: # Defensive: in CI, we always have set_alpn_protocols │ │ 467 │ │ pass │ │ 468 │ │ │ ❱ 469 │ ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname) │ │ 470 │ return ssl_sock │ │ 471 │ │ 472 │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ ca_cert_data = None │ │ │ │ ca_cert_dir = None │ │ │ │ ca_certs = 'C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-pac… │ │ │ │ cert_reqs = None │ │ │ │ certfile = None │ │ │ │ ciphers = None │ │ │ │ context = <ssl.SSLContext object at 0x000001E2958EB020> │ │ │ │ key_password = None │ │ │ │ keyfile = None │ │ │ │ server_hostname = 'huggingface.co' │ │ │ │ sock = <socket.socket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ ssl_context = <ssl.SSLContext object at 0x000001E2958EB020> │ │ │ │ ssl_version = None │ │ │ │ tls_intls = False │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\ssl.py:513 in │ │ _ssl_wrap_socket_impl │ │ │ │ 510 │ │ SSLTransport._validate_ssl_context_for_tls_in_tls(ssl_context) │ │ 511 │ │ return SSLTransport(sock, ssl_context, server_hostname) │ │ 512 │ │ │ ❱ 513 │ return ssl_context.wrap_socket(sock, server_hostname=server_hostname) │ │ 514 │ │ │ │ ╭─────────────────────────────────── locals ───────────────────────────────────╮ │ │ │ server_hostname = 'huggingface.co' │ │ │ │ sock = <socket.socket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ ssl_context = <ssl.SSLContext object at 0x000001E2958EB020> │ │ │ │ tls_in_tls = False │ │ │ ╰──────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\ssl.py:517 in wrap_socket │ │ │ │ 514 │ │ │ │ │ server_hostname=None, session=None): │ │ 515 │ │ # SSLSocket class handles server_hostname encoding before it calls │ │ 516 │ │ # ctx._wrap_socket() │ │ ❱ 517 │ │ return self.sslsocket_class._create( │ │ 518 │ │ │ sock=sock, │ │ 519 │ │ │ server_side=server_side, │ │ 520 │ │ │ do_handshake_on_connect=do_handshake_on_connect, │ │ │ │ ╭─────────────────────────────────────── locals ───────────────────────────────────────╮ │ │ │ do_handshake_on_connect = True │ │ │ │ self = <ssl.SSLContext object at 0x000001E2958EB020> │ │ │ │ server_hostname = 'huggingface.co' │ │ │ │ server_side = False │ │ │ │ session = None │ │ │ │ sock = <socket.socket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ suppress_ragged_eofs = True │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\ssl.py:1108 in _create │ │ │ │ 1105 │ │ │ │ │ if timeout == 0.0: │ │ 1106 │ │ │ │ │ │ # non-blocking │ │ 1107 │ │ │ │ │ │ raise ValueError("do_handshake_on_connect should not be specifie │ │ ❱ 1108 │ │ │ │ │ self.do_handshake() │ │ 1109 │ │ │ except (OSError, ValueError): │ │ 1110 │ │ │ │ self.close() │ │ 1111 │ │ │ │ raise │ │ │ │ ╭─────────────────────────────────────── locals ───────────────────────────────────────╮ │ │ │ class = <class 'ssl.SSLSocket'> │ │ │ │ cls = <class 'ssl.SSLSocket'> │ │ │ │ connected = True │ │ │ │ context = <ssl.SSLContext object at 0x000001E2958EB020> │ │ │ │ do_handshake_on_connect = True │ │ │ │ kwargs = { │ │ │ │ │ 'family': <AddressFamily.AF_INET6: 23>, │ │ │ │ │ 'type': <SocketKind.SOCK_STREAM: 1>, │ │ │ │ │ 'proto': 0, │ │ │ │ │ 'fileno': 3488 │ │ │ │ } │ │ │ │ self = <ssl.SSLSocket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ server_hostname = 'huggingface.co' │ │ │ │ server_side = False │ │ │ │ session = None │ │ │ │ sock = <socket.socket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ sock_timeout = None │ │ │ │ suppress_ragged_eofs = True │ │ │ │ timeout = None │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\ssl.py:1383 in do_handshake │ │ │ │ 1380 │ │ try: │ │ 1381 │ │ │ if timeout == 0.0 and block: │ │ 1382 │ │ │ │ self.settimeout(None) │ │ ❱ 1383 │ │ │ self._sslobj.do_handshake() │ │ 1384 │ │ finally: │ │ 1385 │ │ │ self.settimeout(timeout) │ │ 1386 │ │ │ │ ╭─────────────────────────────── locals ───────────────────────────────╮ │ │ │ block = False │ │ │ │ self = <ssl.SSLSocket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ timeout = None │ │ │ ╰──────────────────────────────────────────────────────────────────────╯ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py:486 in │ │ send │ │ │ │ 483 │ │ │ timeout = TimeoutSauce(connect=timeout, read=timeout) │ │ 484 │ │ │ │ 485 │ │ try: │ │ ❱ 486 │ │ │ resp = conn.urlopen( │ │ 487 │ │ │ │ method=request.method, │ │ 488 │ │ │ │ url=url, │ │ 489 │ │ │ │ body=request.body, │ │ │ │ ╭────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ cert = None │ │ │ │ chunked = False │ │ │ │ conn = <urllib3.connectionpool.HTTPSConnectionPool object at 0x000001E293406E90> │ │ │ │ proxies = OrderedDict() │ │ │ │ request = <PreparedRequest [GET]> │ │ │ │ self = <huggingface_hub.utils._http.UniqueRequestIdAdapter object at 0x000001E2951D02D0> │ │ │ │ stream = False │ │ │ │ timeout = Timeout(connect=None, read=None, total=None) │ │ │ │ url = '/api/models/hkunlp/instructor-xl' │ │ │ │ verify = True │ │ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:8 │ │ 44 in urlopen │ │ │ │ 841 │ │ │ elif isinstance(new_e, (OSError, HTTPException)): │ │ 842 │ │ │ │ new_e = ProtocolError("Connection aborted.", new_e) │ │ 843 │ │ │ │ │ ❱ 844 │ │ │ retries = retries.increment( │ │ 845 │ │ │ │ method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2] │ │ 846 │ │ │ ) │ │ 847 │ │ │ retries.sleep() │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ assert_same_host = False │ │ │ │ body = None │ │ │ │ body_pos = None │ │ │ │ chunked = False │ │ │ │ clean_exit = False │ │ │ │ conn = None │ │ │ │ decode_content = False │ │ │ │ destination_scheme = None │ │ │ │ err = None │ │ │ │ headers = {'user-agent': 'unknown/None; hf_hub/0.17.3; python/3.11.7; │ │ │ │ torch/2.1.2+cu121', 'Accept-Encoding': 'gzip, deflate, zstd', │ │ │ │ 'Accept': '/', 'Connection': 'keep-alive', 'authorization': 'Bearer │ │ │ │ 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', 'X-Amzn-Trace-Id': │ │ │ │ '603c5e5a-181a-4746-bcd4-3884aa53d505'} │ │ │ │ http_tunnel_required = False │ │ │ │ method = 'GET' │ │ │ │ new_e = ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An │ │ │ │ existing connection was forcibly closed by the remote host', None, │ │ │ │ 10054, None)) │ │ │ │ parsed_url = Url( │ │ │ │ │ scheme=None, │ │ │ │ │ auth=None, │ │ │ │ │ host=None, │ │ │ │ │ port=None, │ │ │ │ │ path='/api/models/hkunlp/instructor-xl', │ │ │ │ │ query=None, │ │ │ │ │ fragment=None │ │ │ │ ) │ │ │ │ pool_timeout = None │ │ │ │ preload_content = False │ │ │ │ redirect = False │ │ │ │ release_conn = False │ │ │ │ release_this_conn = True │ │ │ │ response_conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ response_kw = {} │ │ │ │ retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) │ │ │ │ self = <urllib3.connectionpool.HTTPSConnectionPool object at │ │ │ │ 0x000001E293406E90> │ │ │ │ timeout = Timeout(connect=None, read=None, total=None) │ │ │ │ timeout_obj = Timeout(connect=None, read=None, total=None) │ │ │ │ url = '/api/models/hkunlp/instructor-xl' │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\retry.py:470 │ │ in increment │ │ │ │ 467 │ │ elif error and self._is_read_error(error): │ │ 468 │ │ │ # Read retry? │ │ 469 │ │ │ if read is False or method is None or not self._is_method_retryable(method): │ │ ❱ 470 │ │ │ │ raise reraise(type(error), error, _stacktrace) │ │ 471 │ │ │ elif read is not None: │ │ 472 │ │ │ │ read -= 1 │ │ 473 │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ _pool = <urllib3.connectionpool.HTTPSConnectionPool object at │ │ │ │ 0x000001E293406E90> │ │ │ │ _stacktrace = <traceback object at 0x000001E2954CF200> │ │ │ │ cause = 'unknown' │ │ │ │ connect = None │ │ │ │ error = ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An │ │ │ │ existing connection was forcibly closed by the remote host', None, │ │ │ │ 10054, None)) │ │ │ │ method = 'GET' │ │ │ │ other = None │ │ │ │ read = False │ │ │ │ redirect = None │ │ │ │ redirect_location = None │ │ │ │ response = None │ │ │ │ self = Retry(total=0, connect=None, read=False, redirect=None, status=None) │ │ │ │ status = None │ │ │ │ status_count = None │ │ │ │ total = -1 │ │ │ │ url = '/api/models/hkunlp/instructor-xl' │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\util.py:38 in │ │ reraise │ │ │ │ 35 ) -> typing.NoReturn: │ │ 36 │ try: │ │ 37 │ │ if value.traceback is not tb: │ │ ❱ 38 │ │ │ raise value.with_traceback(tb) │ │ 39 │ │ raise value │ │ 40 │ finally: │ │ 41 │ │ value = None # type: ignore[assignment] │ │ │ │ ╭────────────────────── locals ──────────────────────╮ │ │ │ tb = None │ │ │ │ tp = <class 'urllib3.exceptions.ProtocolError'> │ │ │ │ value = None │ │ │ ╰────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:7 │ │ 90 in urlopen │ │ │ │ 787 │ │ │ response_conn = conn if not release_conn else None │ │ 788 │ │ │ │ │ 789 │ │ │ # Make the request on the HTTPConnection object │ │ ❱ 790 │ │ │ response = self._make_request( │ │ 791 │ │ │ │ conn, │ │ 792 │ │ │ │ method, │ │ 793 │ │ │ │ url, │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ assert_same_host = False │ │ │ │ body = None │ │ │ │ body_pos = None │ │ │ │ chunked = False │ │ │ │ clean_exit = False │ │ │ │ conn = None │ │ │ │ decode_content = False │ │ │ │ destination_scheme = None │ │ │ │ err = None │ │ │ │ headers = {'user-agent': 'unknown/None; hf_hub/0.17.3; python/3.11.7; │ │ │ │ torch/2.1.2+cu121', 'Accept-Encoding': 'gzip, deflate, zstd', │ │ │ │ 'Accept': '/', 'Connection': 'keep-alive', 'authorization': 'Bearer │ │ │ │ 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', 'X-Amzn-Trace-Id': │ │ │ │ '603c5e5a-181a-4746-bcd4-3884aa53d505'} │ │ │ │ http_tunnel_required = False │ │ │ │ method = 'GET' │ │ │ │ new_e = ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An │ │ │ │ existing connection was forcibly closed by the remote host', None, │ │ │ │ 10054, None)) │ │ │ │ parsed_url = Url( │ │ │ │ │ scheme=None, │ │ │ │ │ auth=None, │ │ │ │ │ host=None, │ │ │ │ │ port=None, │ │ │ │ │ path='/api/models/hkunlp/instructor-xl', │ │ │ │ │ query=None, │ │ │ │ │ fragment=None │ │ │ │ ) │ │ │ │ pool_timeout = None │ │ │ │ preload_content = False │ │ │ │ redirect = False │ │ │ │ release_conn = False │ │ │ │ release_this_conn = True │ │ │ │ response_conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ response_kw = {} │ │ │ │ retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) │ │ │ │ self = <urllib3.connectionpool.HTTPSConnectionPool object at │ │ │ │ 0x000001E293406E90> │ │ │ │ timeout = Timeout(connect=None, read=None, total=None) │ │ │ │ timeout_obj = Timeout(connect=None, read=None, total=None) │ │ │ │ url = '/api/models/hkunlp/instructor-xl' │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:4 │ │ 91 in _make_request │ │ │ │ 488 │ │ │ │ new_e, (OSError, NewConnectionError, TimeoutError, SSLError) │ │ 489 │ │ │ ) and (conn and conn.proxy and not conn.has_connected_to_proxy): │ │ 490 │ │ │ │ new_e = _wrap_proxy_error(new_e, conn.proxy.scheme) │ │ ❱ 491 │ │ │ raise new_e │ │ 492 │ │ │ │ 493 │ │ # conn.request() calls http.client..request, not the method in │ │ 494 │ │ # urllib3.request. It also calls makefile (recv) on the socket. │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ body = None │ │ │ │ chunked = False │ │ │ │ conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ decode_content = False │ │ │ │ enforce_content_length = True │ │ │ │ headers = {'user-agent': 'unknown/None; hf_hub/0.17.3; python/3.11.7; │ │ │ │ torch/2.1.2+cu121', 'Accept-Encoding': 'gzip, deflate, zstd', │ │ │ │ 'Accept': '/', 'Connection': 'keep-alive', 'authorization': │ │ │ │ 'Bearer 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', 'X-Amzn-Trace-Id': │ │ │ │ '603c5e5a-181a-4746-bcd4-3884aa53d505'} │ │ │ │ method = 'GET' │ │ │ │ new_e = ConnectionResetError(10054, 'An existing connection was forcibly │ │ │ │ closed by the remote host', None, 10054, None) │ │ │ │ preload_content = False │ │ │ │ response_conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ retries = Retry(total=0, connect=None, read=False, redirect=None, │ │ │ │ status=None) │ │ │ │ self = <urllib3.connectionpool.HTTPSConnectionPool object at │ │ │ │ 0x000001E293406E90> │ │ │ │ timeout = Timeout(connect=None, read=None, total=None) │ │ │ │ timeout_obj = Timeout(connect=None, read=None, total=None) │ │ │ │ url = '/api/models/hkunlp/instructor-xl' │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:4 │ │ 67 in _make_request │ │ │ │ 464 │ │ try: │ │ 465 │ │ │ # Trigger any extra validation we need to do. │ │ 466 │ │ │ try: │ │ ❱ 467 │ │ │ │ self._validate_conn(conn) │ │ 468 │ │ │ except (SocketTimeout, BaseSSLError) as e: │ │ 469 │ │ │ │ self._raise_timeout(err=e, url=url, timeout_value=conn.timeout) │ │ 470 │ │ │ │ raise │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ body = None │ │ │ │ chunked = False │ │ │ │ conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ decode_content = False │ │ │ │ enforce_content_length = True │ │ │ │ headers = {'user-agent': 'unknown/None; hf_hub/0.17.3; python/3.11.7; │ │ │ │ torch/2.1.2+cu121', 'Accept-Encoding': 'gzip, deflate, zstd', │ │ │ │ 'Accept': '/*', 'Connection': 'keep-alive', 'authorization': │ │ │ │ 'Bearer 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', 'X-Amzn-Trace-Id': │ │ │ │ '603c5e5a-181a-4746-bcd4-3884aa53d505'} │ │ │ │ method = 'GET' │ │ │ │ new_e = ConnectionResetError(10054, 'An existing connection was forcibly │ │ │ │ closed by the remote host', None, 10054, None) │ │ │ │ preload_content = False │ │ │ │ response_conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ retries = Retry(total=0, connect=None, read=False, redirect=None, │ │ │ │ status=None) │ │ │ │ self = <urllib3.connectionpool.HTTPSConnectionPool object at │ │ │ │ 0x000001E293406E90> │ │ │ │ timeout = Timeout(connect=None, read=None, total=None) │ │ │ │ timeout_obj = Timeout(connect=None, read=None, total=None) │ │ │ │ url = '/api/models/hkunlp/instructor-xl' │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connectionpool.py:1 │ │ 092 in _validate_conn │ │ │ │ 1089 │ │ │ │ 1090 │ │ # Force connect early to allow us to validate the connection. │ │ 1091 │ │ if conn.is_closed: │ │ ❱ 1092 │ │ │ conn.connect() │ │ 1093 │ │ │ │ 1094 │ │ if not conn.is_verified: │ │ 1095 │ │ │ warnings.warn( │ │ │ │ ╭─────────────────────────────────────── locals ────────────────────────────────────────╮ │ │ │ class = <class 'urllib3.connectionpool.HTTPSConnectionPool'> │ │ │ │ conn = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x000001E293406E90> │ │ │ ╰───────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py:642 │ │ in connect │ │ │ │ 639 │ │ │ │ SystemTimeWarning, │ │ 640 │ │ │ ) │ │ 641 │ │ │ │ ❱ 642 │ │ sock_and_verified = _ssl_wrap_socket_and_match_hostname( │ │ 643 │ │ │ sock=sock, │ │ 644 │ │ │ cert_reqs=self.cert_reqs, │ │ 645 │ │ │ ssl_version=self.ssl_version, │ │ │ │ ╭────────────────────────────────────── locals ───────────────────────────────────────╮ │ │ │ is_time_off = False │ │ │ │ self = <urllib3.connection.HTTPSConnection object at 0x000001E2959BBE90> │ │ │ │ server_hostname = 'huggingface.co' │ │ │ │ sock = <socket.socket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ tls_in_tls = False │ │ │ ╰─────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\connection.py:783 │ │ in _ssl_wrap_socket_and_match_hostname │ │ │ │ 780 │ │ if is_ipaddress(normalized): │ │ 781 │ │ │ server_hostname = normalized │ │ 782 │ │ │ ❱ 783 │ ssl_sock = ssl_wrap_socket( │ │ 784 │ │ sock=sock, │ │ 785 │ │ keyfile=key_file, │ │ 786 │ │ certfile=cert_file, │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ assert_fingerprint = None │ │ │ │ assert_hostname = None │ │ │ │ ca_cert_data = None │ │ │ │ ca_cert_dir = None │ │ │ │ ca_certs = 'C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site… │ │ │ │ cert_file = None │ │ │ │ cert_reqs = 'CERT_REQUIRED' │ │ │ │ context = <ssl.SSLContext object at 0x000001E2958EB020> │ │ │ │ default_ssl_context = True │ │ │ │ key_file = None │ │ │ │ key_password = None │ │ │ │ normalized = 'huggingface.co' │ │ │ │ server_hostname = 'huggingface.co' │ │ │ │ sock = <socket.socket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ ssl_context = None │ │ │ │ ssl_maximum_version = None │ │ │ │ ssl_minimum_version = None │ │ │ │ ssl_version = None │ │ │ │ tls_intls = False │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\ssl.py:469 in │ │ ssl_wrap_socket │ │ │ │ 466 │ except NotImplementedError: # Defensive: in CI, we always have set_alpn_protocols │ │ 467 │ │ pass │ │ 468 │ │ │ ❱ 469 │ ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname) │ │ 470 │ return ssl_sock │ │ 471 │ │ 472 │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ ca_cert_data = None │ │ │ │ ca_cert_dir = None │ │ │ │ ca_certs = 'C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-pac… │ │ │ │ cert_reqs = None │ │ │ │ certfile = None │ │ │ │ ciphers = None │ │ │ │ context = <ssl.SSLContext object at 0x000001E2958EB020> │ │ │ │ key_password = None │ │ │ │ keyfile = None │ │ │ │ server_hostname = 'huggingface.co' │ │ │ │ sock = <socket.socket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ ssl_context = <ssl.SSLContext object at 0x000001E2958EB020> │ │ │ │ ssl_version = None │ │ │ │ tls_intls = False │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\urllib3\util\ssl.py:513 in │ │ _ssl_wrap_socket_impl │ │ │ │ 510 │ │ SSLTransport._validate_ssl_context_for_tls_in_tls(ssl_context) │ │ 511 │ │ return SSLTransport(sock, ssl_context, server_hostname) │ │ 512 │ │ │ ❱ 513 │ return ssl_context.wrap_socket(sock, server_hostname=server_hostname) │ │ 514 │ │ │ │ ╭─────────────────────────────────── locals ───────────────────────────────────╮ │ │ │ server_hostname = 'huggingface.co' │ │ │ │ sock = <socket.socket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ ssl_context = <ssl.SSLContext object at 0x000001E2958EB020> │ │ │ │ tls_in_tls = False │ │ │ ╰──────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\ssl.py:517 in wrap_socket │ │ │ │ 514 │ │ │ │ │ server_hostname=None, session=None): │ │ 515 │ │ # SSLSocket class handles server_hostname encoding before it calls │ │ 516 │ │ # ctx._wrap_socket() │ │ ❱ 517 │ │ return self.sslsocket_class._create( │ │ 518 │ │ │ sock=sock, │ │ 519 │ │ │ server_side=server_side, │ │ 520 │ │ │ do_handshake_on_connect=do_handshake_on_connect, │ │ │ │ ╭─────────────────────────────────────── locals ───────────────────────────────────────╮ │ │ │ do_handshake_on_connect = True │ │ │ │ self = <ssl.SSLContext object at 0x000001E2958EB020> │ │ │ │ server_hostname = 'huggingface.co' │ │ │ │ server_side = False │ │ │ │ session = None │ │ │ │ sock = <socket.socket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ suppress_ragged_eofs = True │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\ssl.py:1108 in _create │ │ │ │ 1105 │ │ │ │ │ if timeout == 0.0: │ │ 1106 │ │ │ │ │ │ # non-blocking │ │ 1107 │ │ │ │ │ │ raise ValueError("do_handshake_on_connect should not be specifie │ │ ❱ 1108 │ │ │ │ │ self.do_handshake() │ │ 1109 │ │ │ except (OSError, ValueError): │ │ 1110 │ │ │ │ self.close() │ │ 1111 │ │ │ │ raise │ │ │ │ ╭─────────────────────────────────────── locals ───────────────────────────────────────╮ │ │ │ class = <class 'ssl.SSLSocket'> │ │ │ │ cls = <class 'ssl.SSLSocket'> │ │ │ │ connected = True │ │ │ │ context = <ssl.SSLContext object at 0x000001E2958EB020> │ │ │ │ do_handshake_on_connect = True │ │ │ │ kwargs = { │ │ │ │ │ 'family': <AddressFamily.AF_INET6: 23>, │ │ │ │ │ 'type': <SocketKind.SOCK_STREAM: 1>, │ │ │ │ │ 'proto': 0, │ │ │ │ │ 'fileno': 3488 │ │ │ │ } │ │ │ │ self = <ssl.SSLSocket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ server_hostname = 'huggingface.co' │ │ │ │ server_side = False │ │ │ │ session = None │ │ │ │ sock = <socket.socket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ sock_timeout = None │ │ │ │ suppress_ragged_eofs = True │ │ │ │ timeout = None │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\ssl.py:1383 in do_handshake │ │ │ │ 1380 │ │ try: │ │ 1381 │ │ │ if timeout == 0.0 and block: │ │ 1382 │ │ │ │ self.settimeout(None) │ │ ❱ 1383 │ │ │ self._sslobj.do_handshake() │ │ 1384 │ │ finally: │ │ 1385 │ │ │ self.settimeout(timeout) │ │ 1386 │ │ │ │ ╭─────────────────────────────── locals ───────────────────────────────╮ │ │ │ block = False │ │ │ │ self = <ssl.SSLSocket [closed] fd=-1, family=23, type=1, proto=0> │ │ │ │ timeout = None │ │ │ ╰──────────────────────────────────────────────────────────────────────╯ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ ProtocolError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

During handling of the above exception, another exception occurred:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\chatdocs\main.py:26 in │ │ download │ │ │ │ 23 │ from .download import download │ │ 24 │ │ │ 25 │ config = get_config(config) │ │ ❱ 26 │ download(config=config) │ │ 27 │ │ 28 │ │ 29 @app.command() │ │ │ │ ╭───────────────────────────────── locals ─────────────────────────────────╮ │ │ │ config = { │ │ │ │ │ 'embeddings': { │ │ │ │ │ │ 'model': 'hkunlp/instructor-xl', │ │ │ │ │ │ 'model_kwargs': {'device': 'cuda'} │ │ │ │ │ }, │ │ │ │ │ 'llm': 'ctransformers', │ │ │ │ │ 'ctransformers': { │ │ │ │ │ │ 'model': 'TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF', │ │ │ │ │ │ 'model_file': 'tinyllama-1.1b-chat-v1.0.Q5_K_M.gguf', │ │ │ │ │ │ 'model_type': 'llama', │ │ │ │ │ │ 'config': { │ │ │ │ │ │ │ 'context_length': 1024, │ │ │ │ │ │ │ 'max_new_tokens': 256, │ │ │ │ │ │ │ 'gpu_layers': 4 │ │ │ │ │ │ } │ │ │ │ │ }, │ │ │ │ │ 'huggingface': { │ │ │ │ │ │ 'model': 'TheBloke/Wizard-Vicuna-7B-Uncensored-HF', │ │ │ │ │ │ 'pipeline_kwargs': {'max_new_tokens': 256} │ │ │ │ │ }, │ │ │ │ │ 'gptq': { │ │ │ │ │ │ 'model': 'TheBloke/Wizard-Vicuna-7B-Uncensored-GPTQ', │ │ │ │ │ │ 'model_file': 'model.safetensors', │ │ │ │ │ │ 'pipeline_kwargs': {'max_new_tokens': 256} │ │ │ │ │ }, │ │ │ │ │ 'download': False, │ │ │ │ │ 'host': 'localhost', │ │ │ │ │ 'port': 5000, │ │ │ │ │ 'auth': False, │ │ │ │ │ 'chroma': { │ │ │ │ │ │ 'persist_directory': 'db', │ │ │ │ │ │ 'chroma_db_impl': 'duckdb+parquet', │ │ │ │ │ │ 'anonymized_telemetry': False │ │ │ │ │ }, │ │ │ │ │ ... +1 │ │ │ │ } │ │ │ │ download = <function download at 0x000001E2B63EE840> │ │ │ ╰──────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\chatdocs\download.py:9 in │ │ download │ │ │ │ 6 │ │ 7 def download(config: Dict[str, Any]) -> None: │ │ 8 │ config = {config, "download": True} │ │ ❱ 9 │ get_embeddings(config) │ │ 10 │ get_llm(config) │ │ 11 │ │ │ │ ╭──────────────────────────────── locals ────────────────────────────────╮ │ │ │ config = { │ │ │ │ │ 'embeddings': { │ │ │ │ │ │ 'model': 'hkunlp/instructor-xl', │ │ │ │ │ │ 'model_kwargs': {'device': 'cuda'} │ │ │ │ │ }, │ │ │ │ │ 'llm': 'ctransformers', │ │ │ │ │ 'ctransformers': { │ │ │ │ │ │ 'model': 'TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF', │ │ │ │ │ │ 'model_file': 'tinyllama-1.1b-chat-v1.0.Q5_K_M.gguf', │ │ │ │ │ │ 'model_type': 'llama', │ │ │ │ │ │ 'config': { │ │ │ │ │ │ │ 'context_length': 1024, │ │ │ │ │ │ │ 'max_new_tokens': 256, │ │ │ │ │ │ │ 'gpu_layers': 4 │ │ │ │ │ │ } │ │ │ │ │ }, │ │ │ │ │ 'huggingface': { │ │ │ │ │ │ 'model': 'TheBloke/Wizard-Vicuna-7B-Uncensored-HF', │ │ │ │ │ │ 'pipeline_kwargs': {'max_new_tokens': 256} │ │ │ │ │ }, │ │ │ │ │ 'gptq': { │ │ │ │ │ │ 'model': 'TheBloke/Wizard-Vicuna-7B-Uncensored-GPTQ', │ │ │ │ │ │ 'model_file': 'model.safetensors', │ │ │ │ │ │ 'pipeline_kwargs': {'max_new_tokens': 256} │ │ │ │ │ }, │ │ │ │ │ 'download': True, │ │ │ │ │ 'host': 'localhost', │ │ │ │ │ 'port': 5000, │ │ │ │ │ 'auth': False, │ │ │ │ │ 'chroma': { │ │ │ │ │ │ 'persist_directory': 'db', │ │ │ │ │ │ 'chroma_db_impl': 'duckdb+parquet', │ │ │ │ │ │ 'anonymized_telemetry': False │ │ │ │ │ }, │ │ │ │ │ ... +1 │ │ │ │ } │ │ │ ╰────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\chatdocs\embeddings.py:14 │ │ in get_embeddings │ │ │ │ 11 │ │ Provider = HuggingFaceInstructEmbeddings │ │ 12 │ else: │ │ 13 │ │ Provider = HuggingFaceEmbeddings │ │ ❱ 14 │ return Provider(config) │ │ 15 │ │ │ │ ╭─────────────────────────────────────── locals ────────────────────────────────────────╮ │ │ │ config = {'model_kwargs': {'device': 'cuda'}, 'model_name': 'hkunlp/instructor-xl'} │ │ │ │ Provider = <class 'langchain.embeddings.huggingface.HuggingFaceInstructEmbeddings'> │ │ │ ╰───────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\embeddings\huggin │ │ gface.py:150 in init │ │ │ │ 147 │ │ try: │ │ 148 │ │ │ from InstructorEmbedding import INSTRUCTOR │ │ 149 │ │ │ │ │ ❱ 150 │ │ │ self.client = INSTRUCTOR( │ │ 151 │ │ │ │ self.model_name, cache_folder=self.cache_folder, **self.model_kwargs │ │ 152 │ │ │ ) │ │ 153 │ │ except ImportError as e: │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ class = <class 'langchain.embeddings.huggingface.HuggingFaceInstructEmbeddings'> │ │ │ │ INSTRUCTOR = <class 'InstructorEmbedding.instructor.INSTRUCTOR'> │ │ │ │ kwargs = {'model_kwargs': {'device': 'cuda'}, 'model_name': 'hkunlp/instructor-xl'} │ │ │ │ self = HuggingFaceInstructEmbeddings( │ │ │ │ │ client=None, │ │ │ │ │ model_name='hkunlp/instructor-xl', │ │ │ │ │ cache_folder=None, │ │ │ │ │ model_kwargs={'device': 'cuda'}, │ │ │ │ │ encode_kwargs={}, │ │ │ │ │ embed_instruction='Represent the document for retrieval: ', │ │ │ │ │ query_instruction='Represent the question for retrieving supporting │ │ │ │ documents: ' │ │ │ │ ) │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\sentence_transformers\Sente │ │ nceTransformer.py:87 in init │ │ │ │ 84 │ │ │ │ │ │ 85 │ │ │ │ if not os.path.exists(os.path.join(model_path, 'modules.json')): │ │ 86 │ │ │ │ │ # Download from hub with caching │ │ ❱ 87 │ │ │ │ │ snapshot_download(model_name_or_path, │ │ 88 │ │ │ │ │ │ │ │ │ │ cache_dir=cache_folder, │ │ 89 │ │ │ │ │ │ │ │ │ │ library_name='sentence-transformers', │ │ 90 │ │ │ │ │ │ │ │ │ │ library_version=version, │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ class = <class │ │ │ │ 'sentence_transformers.SentenceTransformer.SentenceTransformer'> │ │ │ │ _get_torch_home = <function _get_torch_home at 0x000001E2E639B920> │ │ │ │ basic_transformer_models = [ │ │ │ │ │ 'albert-base-v1', │ │ │ │ │ 'albert-base-v2', │ │ │ │ │ 'albert-large-v1', │ │ │ │ │ 'albert-large-v2', │ │ │ │ │ 'albert-xlarge-v1', │ │ │ │ │ 'albert-xlarge-v2', │ │ │ │ │ 'albert-xxlarge-v1', │ │ │ │ │ 'albert-xxlarge-v2', │ │ │ │ │ 'bert-base-cased-finetuned-mrpc', │ │ │ │ │ 'bert-base-cased', │ │ │ │ │ ... +58 │ │ │ │ ] │ │ │ │ cache_folder = 'C:\Users\j/.cache\torch\sentence_transformers' │ │ │ │ device = 'cuda' │ │ │ │ model_name_or_path = 'hkunlp/instructor-xl' │ │ │ │ model_path = 'C:\Users\j/.cache\torch\sentence_transformers\hkunlp_instr… │ │ │ │ modules = None │ │ │ │ self = <repr-error "'INSTRUCTOR' object has no attribute '_modules'"> │ │ │ │ torch_cache_home = 'C:\Users\j/.cache\torch' │ │ │ │ use_auth_token = None │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\sentence_transformers\util. │ │ py:442 in snapshot_download │ │ │ │ 439 │ elif use_auth_token: │ │ 440 │ │ token = HfFolder.get_token() │ │ 441 │ │ │ ❱ 442 │ model_info = _api.model_info(repo_id=repo_id, revision=revision, token=token) │ │ 443 │ │ │ 444 │ storage_folder = os.path.join( │ │ 445 │ │ cache_dir, repoid.replace("/", "") │ │ │ │ ╭─────────────────────────────────── locals ────────────────────────────────────╮ │ │ │ _api = <huggingface_hub.hf_api.HfApi object at 0x000001E2B71B67D0> │ │ │ │ cache_dir = 'C:\Users\j/.cache\torch\sentence_transformers' │ │ │ │ ignore_files = ['flax_model.msgpack', 'rust_model.ot', 'tf_model.h5'] │ │ │ │ library_name = 'sentence-transformers' │ │ │ │ library_version = '2.2.2' │ │ │ │ repo_id = 'hkunlp/instructor-xl' │ │ │ │ revision = None │ │ │ │ token = None │ │ │ │ use_auth_token = None │ │ │ │ user_agent = None │ │ │ ╰───────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\huggingface_hub\utils_vali │ │ dators.py:118 in _inner_fn │ │ │ │ 115 │ │ if check_use_auth_token: │ │ 116 │ │ │ kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.name, has_token=ha │ │ 117 │ │ │ │ ❱ 118 │ │ return fn(*args, kwargs) │ │ 119 │ │ │ 120 │ return _inner_fn # type: ignore │ │ 121 │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ arg_name = 'token' │ │ │ │ arg_value = None │ │ │ │ args = (<huggingface_hub.hf_api.HfApi object at 0x000001E2B71B67D0>,) │ │ │ │ check_use_auth_token = True │ │ │ │ fn = <function HfApi.model_info at 0x000001E294C88EA0> │ │ │ │ has_token = False │ │ │ │ kwargs = {'repo_id': 'hkunlp/instructor-xl', 'revision': None, 'token': None} │ │ │ │ signature = <Signature (self, repo_id: 'str', *, revision: 'Optional[str]' = │ │ │ │ None, timeout: 'Optional[float]' = None, securityStatus: │ │ │ │ 'Optional[bool]' = None, files_metadata: 'bool' = False, token: │ │ │ │ 'Optional[Union[bool, str]]' = None) -> 'ModelInfo'> │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\huggingface_hub\hf_api.py:1 │ │ 697 in model_info │ │ │ │ 1694 │ │ │ params["securityStatus"] = True │ │ 1695 │ │ if files_metadata: │ │ 1696 │ │ │ params["blobs"] = True │ │ ❱ 1697 │ │ r = get_session().get(path, headers=headers, timeout=timeout, params=params) │ │ 1698 │ │ hf_raise_for_status(r) │ │ 1699 │ │ d = r.json() │ │ 1700 │ │ return ModelInfo(d) │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ files_metadata = False │ │ │ │ headers = { │ │ │ │ │ 'user-agent': 'unknown/None; hf_hub/0.17.3; python/3.11.7; │ │ │ │ torch/2.1.2+cu121', │ │ │ │ │ 'authorization': 'Bearer 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' │ │ │ │ } │ │ │ │ params = {} │ │ │ │ path = 'https://huggingface.co/api/models/hkunlp/instructor-xl' │ │ │ │ repo_id = 'hkunlp/instructor-xl' │ │ │ │ revision = None │ │ │ │ securityStatus = None │ │ │ │ self = <huggingface_hub.hf_api.HfApi object at 0x000001E2B71B67D0> │ │ │ │ timeout = None │ │ │ │ token = None │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py:602 in │ │ get │ │ │ │ 599 │ │ """ │ │ 600 │ │ │ │ 601 │ │ kwargs.setdefault("allow_redirects", True) │ │ ❱ 602 │ │ return self.request("GET", url, kwargs) │ │ 603 │ │ │ 604 │ def options(self, url, kwargs): │ │ 605 │ │ r"""Sends a OPTIONS request. Returns :class:Response object. │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ kwargs = { │ │ │ │ │ 'headers': { │ │ │ │ │ │ 'user-agent': 'unknown/None; hf_hub/0.17.3; python/3.11.7; │ │ │ │ torch/2.1.2+cu121', │ │ │ │ │ │ 'authorization': 'Bearer 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' │ │ │ │ │ }, │ │ │ │ │ 'timeout': None, │ │ │ │ │ 'params': {}, │ │ │ │ │ 'allow_redirects': True │ │ │ │ } │ │ │ │ self = <requests.sessions.Session object at 0x000001E2951B0810> │ │ │ │ url = 'https://huggingface.co/api/models/hkunlp/instructor-xl' │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py:589 in │ │ request │ │ │ │ 586 │ │ │ "allow_redirects": allow_redirects, │ │ 587 │ │ } │ │ 588 │ │ send_kwargs.update(settings) │ │ ❱ 589 │ │ resp = self.send(prep, send_kwargs) │ │ 590 │ │ │ │ 591 │ │ return resp │ │ 592 │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ allow_redirects = True │ │ │ │ auth = None │ │ │ │ cert = None │ │ │ │ cookies = None │ │ │ │ data = None │ │ │ │ files = None │ │ │ │ headers = { │ │ │ │ │ 'user-agent': 'unknown/None; hf_hub/0.17.3; python/3.11.7; │ │ │ │ torch/2.1.2+cu121', │ │ │ │ │ 'authorization': 'Bearer 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' │ │ │ │ } │ │ │ │ hooks = None │ │ │ │ json = None │ │ │ │ method = 'GET' │ │ │ │ params = {} │ │ │ │ prep = <PreparedRequest [GET]> │ │ │ │ proxies = {} │ │ │ │ req = <Request [GET]> │ │ │ │ self = <requests.sessions.Session object at 0x000001E2951B0810> │ │ │ │ send_kwargs = { │ │ │ │ │ 'timeout': None, │ │ │ │ │ 'allow_redirects': True, │ │ │ │ │ 'proxies': OrderedDict(), │ │ │ │ │ 'stream': False, │ │ │ │ │ 'verify': True, │ │ │ │ │ 'cert': None │ │ │ │ } │ │ │ │ settings = {'proxies': OrderedDict(), 'stream': False, 'verify': True, 'cert': None} │ │ │ │ stream = None │ │ │ │ timeout = None │ │ │ │ url = 'https://huggingface.co/api/models/hkunlp/instructor-xl' │ │ │ │ verify = None │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\sessions.py:703 in │ │ send │ │ │ │ 700 │ │ start = preferred_clock() │ │ 701 │ │ │ │ 702 │ │ # Send the request │ │ ❱ 703 │ │ r = adapter.send(request, *kwargs) │ │ 704 │ │ │ │ 705 │ │ # Total elapsed time of the request (approximately) │ │ 706 │ │ elapsed = preferred_clock() - start │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ adapter = <huggingface_hub.utils._http.UniqueRequestIdAdapter object at │ │ │ │ 0x000001E2951D02D0> │ │ │ │ allow_redirects = True │ │ │ │ hooks = {'response': []} │ │ │ │ kwargs = { │ │ │ │ │ 'timeout': None, │ │ │ │ │ 'proxies': OrderedDict(), │ │ │ │ │ 'stream': False, │ │ │ │ │ 'verify': True, │ │ │ │ │ 'cert': None │ │ │ │ } │ │ │ │ request = <PreparedRequest [GET]> │ │ │ │ self = <requests.sessions.Session object at 0x000001E2951B0810> │ │ │ │ start = 705.0925496 │ │ │ │ stream = False │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\huggingface_hub\utils_http │ │ .py:63 in send │ │ │ │ 60 │ def send(self, request: PreparedRequest, args, kwargs) -> Response: │ │ 61 │ │ """Catch any RequestException to append request id to the error message for debu │ │ 62 │ │ try: │ │ ❱ 63 │ │ │ return super().send(request, *args, **kwargs) │ │ 64 │ │ except requests.RequestException as e: │ │ 65 │ │ │ request_id = request.headers.get(X_AMZN_TRACE_ID) │ │ 66 │ │ │ if request_id is not None: │ │ │ │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ class = <class 'huggingface_hub.utils._http.UniqueRequestIdAdapter'> │ │ │ │ args = () │ │ │ │ kwargs = { │ │ │ │ │ 'timeout': None, │ │ │ │ │ 'proxies': OrderedDict(), │ │ │ │ │ 'stream': False, │ │ │ │ │ 'verify': True, │ │ │ │ │ 'cert': None │ │ │ │ } │ │ │ │ request = <PreparedRequest [GET]> │ │ │ │ request_id = '603c5e5a-181a-4746-bcd4-3884aa53d505' │ │ │ │ self = <huggingface_hub.utils._http.UniqueRequestIdAdapter object at │ │ │ │ 0x000001E2951D02D0> │ │ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ C:\Users\j\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests\adapters.py:501 in │ │ send │ │ │ │ 498 │ │ │ ) │ │ 499 │ │ │ │ 500 │ │ except (ProtocolError, OSError) as err: │ │ ❱ 501 │ │ │ raise ConnectionError(err, request=request) │ │ 502 │ │ │ │ 503 │ │ except MaxRetryError as e: │ │ 504 │ │ │ if isinstance(e.reason, ConnectTimeoutError): │ │ │ │ ╭────────────────────────────────────────── locals ───────────────────────────────────────────╮ │ │ │ cert = None │ │ │ │ chunked = False │ │ │ │ conn = <urllib3.connectionpool.HTTPSConnectionPool object at 0x000001E293406E90> │ │ │ │ proxies = OrderedDict() │ │ │ │ request = <PreparedRequest [GET]> │ │ │ │ self = <huggingface_hub.utils._http.UniqueRequestIdAdapter object at 0x000001E2951D02D0> │ │ │ │ stream = False │ │ │ │ timeout = Timeout(connect=None, read=None, total=None) │ │ │ │ url = '/api/models/hkunlp/instructor-xl' │ │ │ │ verify = True │ │ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────╯ │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ ConnectionError: (ProtocolError('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)), '(Request ID: 603c5e5a-181a-4746-bcd4-3884aa53d505)')

[davedgd]

For the time being, this might fix the issue you all have noted (at least for GGUF models):

pip install langchain==0.0.354 sentence-transformers==2.2.2 chatdocs

Here's a turnkey demo script you can try as well that's working as of right now:

ChatDocs.ipynb