zhongshsh
commented
5 months ago - Our model is based on
Qwen/Qwen-VL-Chat, and using the int4 version ofQwen-VLdirectly might compromise the quality of responses. - This error may be caused by creating a public link. We have updated the default args of
gradio_demo.py, setting theshareconfig of gradio demo toFalse. You can pull the code and try to run the demo again.
I used Qwen-chat vl int4, but there's no result generate on my 3060. And also there's no explicit error. Here is the log message: (base) F:\CLoT>python inference.py CUDA extension not installed. CUDA extension not installed. C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\transformers\modeling_utils.py:4225: FutureWarning:
_is_quantized_training_enabledis going to be deprecated in transformers 4.39.0. Please usemodel.hf_quantizer.is_trainableinstead warnings.warn( Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 5/5 [00:04<00:00, 1.02it/s]And also the gradio has some timeout error : Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████| 5/5 [00:05<00:00, 1.02s/it] Running on local URL: http://127.0.0.1:2333 Exception in thread Thread-4 (_do_normal_analytics_request): Traceback (most recent call last): File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 69, in map_httpcore_exceptions yield File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 233, in handle_request resp = self._pool.handle_request(req) File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_sync\connection_pool.py", line 216, in handle_request raise exc from None File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_sync\connection_pool.py", line 196, in handle_request response = connection.handle_request( File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_sync\connection.py", line 99, in handle_request raise exc File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_sync\connection.py", line 76, in handle_request stream = self._connect(request) File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_sync\connection.py", line 122, in _connect stream = self._network_backend.connect_tcp(**kwargs) File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_backends\sync.py", line 205, in connect_tcp with map_exceptions(exc_map): File "d:\miniconda3\lib\contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpcore_exceptions.py", line 14, in map_exceptions raise to_exc(exc) from exc httpcore.ConnectTimeout: timed out
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "d:\miniconda3\lib\threading.py", line 1016, in _bootstrap_inner self.run() File "d:\miniconda3\lib\threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\gradio\analytics.py", line 63, in _do_normal_analytics_request httpx.post(url, data=data, timeout=5) File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_api.py", line 319, in post return request( File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_api.py", line 106, in request return client.request( File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 827, in request return self.send(request, auth=auth, follow_redirects=follow_redirects) File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 914, in send response = self._send_handling_auth( File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 942, in _send_handling_auth response = self._send_handling_redirects( File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 979, in _send_handling_redirects response = self._send_single_request(request) File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_client.py", line 1015, in _send_single_request response = transport.handle_request(request) File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 232, in handle_request with map_httpcore_exceptions(): File "d:\miniconda3\lib\contextlib.py", line 153, in exit self.gen.throw(typ, value, traceback) File "C:\Users\Administrator\AppData\Roaming\Python\Python310\site-packages\httpx_transports\default.py", line 86, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectTimeout: timed out