search

quic / ai-hub-models

The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.
https://aihub.qualcomm.com
BSD 3-Clause "New" or "Revised" License
438 stars 60 forks source link

error occured when I run llama_v2_7b_chat_quantized #76

Closed yolanda1224git closed 2 months ago

yolanda1224git commented 2 months ago

Describe the bug I follow the guide and run commands: python -m qai_hub_models.models.llama_v2_7b_chat_quantized.export An error is always reported, log is ConnectionResetError: [WinError 10054] 'an existing connection was forcibly closed by the remote host'

It always occured when run ”uploading model“. How can I fix it?

When I run small model, such as qai_hub_models.models.ffnet_40s.export , and there is alway OK , no error occured.

To Reproduce Steps to reproduce the behavior:

  1. conda activate qai_hub
  2. pip install qai-hub
  3. qai-hub configure --api_token xxx
  4. pip install huggingface-hub==0.23.1
  5. huggingface-cli login
  6. pip3 install "qai-hub-models[llama_v2_7b_chat_quantized]"
  7. python -m qai_hub_models.models.llama_v2_7b_chat_quantized.export

Expected behavior I hope the command can run successfully. python -m qai_hub_models.models.llama_v2_7b_chat_quantized.export

Stack trace If applicable, add screenshots to help explain your problem.

Host configuration:

Additional context Add any other context about the problem here.

yolanda1224git commented 2 months ago

Last time，I submit 3 compile jobs successfully and occur the error when submit the 4th jobs. It occured when run ”uploading model“.

yolanda1224git commented 2 months ago

Detailed log are as follows:

Exporting model to ONNX with AIMET encodings Optimizing model PromptProcessor_4Quantized to run on-device Uploading model: 7%|███████▏ | 224M/3.16G [01:56<18:57:01, 46.3kB/s]Traceback (most recent call last): File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\site-packages\urllib3\connectionpool.py", line 715, in urlopen httplib_response = self._makerequest( File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\site-packages\urllib3\connectionpool.py", line 416, in _make_request conn.request(method, url, httplib_requestkw) File "C:\Users\hiai\miniconda3\envs\qaihub\lib\site-packages\urllib3\connection.py", line 244, in request super(HTTPConnection, self).request(method, url, body=body, headers=headers) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\http\client.py", line 1256, in request self._send_request(method, url, body, headers, encodechunked) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\http\client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encodechunked) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\http\client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encodechunked) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\http\client.py", line 1050, in _sendoutput self.send(chunk) File "C:\Users\hiai\miniconda3\envs\qaihub\lib\http\client.py", line 972, in send self.sock.sendall(data) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\ssl.py", line 1237, in sendall v = self.send(byteview[count:]) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\ssl.py", line 1206, in send return self._sslobj.write(data) ConnectionResetError: [WinError 10054] an existing connection was forcibly closed by the remote host**

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\hiai_\miniconda3\envs\qaihub\lib\site-packages\requests\adapters.py", line 667, in send resp = conn.urlopen( File "C:\Users\hiai\miniconda3\envs\qaihub\lib\site-packages\urllib3\connectionpool.py", line 799, in urlopen retries = retries.increment( File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\site-packages\urllib3\util\retry.py", line 550, in increment raise six.reraise(type(error), error, stacktrace) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\site-packages\urllib3\packages\six.py", line 769, in reraise raise value.withtraceback(tb) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\site-packages\urllib3\connectionpool.py", line 715, in urlopen httplib_response = self._makerequest( File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\site-packages\urllib3\connectionpool.py", line 416, in _make_request conn.request(method, url, **httplib_requestkw) File "C:\Users\hiai\miniconda3\envs\qaihub\lib\site-packages\urllib3\connection.py", line 244, in request super(HTTPConnection, self).request(method, url, body=body, headers=headers) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\http\client.py", line 1256, in request self._send_request(method, url, body, headers, encodechunked) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\http\client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encodechunked) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\http\client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encodechunked) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\http\client.py", line 1050, in _sendoutput self.send(chunk) File "C:\Users\hiai\miniconda3\envs\qaihub\lib\http\client.py", line 972, in send self.sock.sendall(data) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\ssl.py", line 1237, in sendall v = self.send(byteview[count:]) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\ssl.py", line 1206, in send return self._sslobj.write(data) urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(10054, 'an existing connection was forcibly closed by the remote host.', None, 10054, None))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "./export.py", line 285, in main() File "./export.py", line 281, in main export_model(*vars(args)) File "./export.py", line 161, in export_model submitted_compile_job = hub.submit_compilejob( File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\site-packages\qai_hub\client.py", line 3099, in submit_compile_job model = self._upload_model(model, model_type=modeltype) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\site-packages\qai_hub\client.py", line 2429, in _upload_model res_pb = _apicall( File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\site-packages\qai_hub\client.py", line 512, in _api_call return api_func(args, kwargs) File "C:\Users\hiai_\miniconda3\envs\qai_hub\lib\site-packages\qai_hub\public_rest_api.py", line 1265, in create_and_upload_model _uploadasset( File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\site-packages\qai_hub\public_rest_api.py", line 1068, in _upload_asset response = createsession().post( File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\site-packages\requests\sessions.py", line 637, in post return self.request("POST", url, data=data, json=json, kwargs) File "C:\Users\hiai_\miniconda3\envs\qai_hub\lib\site-packages\requests\sessions.py", line 589, in request resp = self.send(prep, sendkwargs) File "C:\Users\hiai\miniconda3\envs\qai_hub\lib\site-packages\requests\sessions.py", line 703, in send r = adapter.send(request, kwargs) File "C:\Users\hiai_\miniconda3\envs\qai_hub\lib\site-packages\requests\adapters.py", line 682, in send raise ConnectionError(err, request=request) requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'an existing connection was forcibly closed by the remote host.', None, 10054, None)) Uploading model: 7%|███████▎ | 225M/3.16G [01:56<26:03, 2.02MB/s]

bhushan23 commented 2 months ago

HI @yolanda1224git could you please use --component to export each model separately? that way you should be able to go through each model export and get compiled asset. e.g. 

python -m qai_hub_models.models.llama_v2_7b_chat_quantized.export --components PromptProcessor_1_Quantized
python -m qai_hub_models.models.llama_v2_7b_chat_quantized.export --components PromptProcessor_2_Quantized
...

You can find list of components with

python -m qai_hub_models.models.llama_v2_7b_chat_quantized.export -h

Later you can use model_ids of these exported model to run on-device demo. We will look into connection issue meanwhile.

yolanda1224git commented 2 months ago

thank u so much