search

quic / ai-hub-models

The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.
https://aihub.qualcomm.com
BSD 3-Clause "New" or "Revised" License
438 stars 60 forks source link

[BUG] openAI-Clip demo failed on cuda machine #21

Closed mestrona-3 closed 5 months ago

mestrona-3 commented 6 months ago

On AI Hub Models Slack, Hu Eric shared that openAI-Clip demo failed for him. https://qualcomm-ai-hub.slack.com/archives/C06LT6T3REY/p1709470194079099

Kory initially took a look. It seems like this has something to do with CUDA availability. This bug is being filed to look into the initially reported bug.

(qai_hub) a19284@njai-ubuntu:~/workspace/qai-hub-clip$ python -m qai_hub_models.models.openai_clip.demo Traceback (most recent call last): File "/home/a19284/mambaforge/envs/qai_hub/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/a19284/mambaforge/envs/qai_hub/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/a19284/mambaforge/envs/qai_hub/lib/python3.8/site-packages/qai_hub_models/models/openai_clip/demo.py", line 98, in main() File "/home/a19284/mambaforge/envs/qai_hub/lib/python3.8/site-packages/qai_hub_models/models/openai_clip/demo.py", line 72, in main predictions = app.predict_similarity(images, text).flatten() File "/home/a19284/mambaforge/envs/qai_hub/lib/python3.8/site-packages/qai_hub_models/models/openai_clip/app.py", line 64, in predict_similarity image_features = self.image_encoder(image) File "/home/a19284/mambaforge/envs/qai_hub/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(*input, *kwargs) File "/home/a19284/mambaforge/envs/qai_hub/lib/python3.8/site-packages/qai_hub_models/models/openai_clip/model.py", line 134, in forward image_features = self.net.encode_image(image) File "/home/a19284/.qaihm/models/openai_clip/v1/openai_CLIP_git/clip/model.py", line 341, in encode_image return self.visual(image.type(self.dtype)) File "/home/a19284/mambaforge/envs/qai_hub/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(input, kwargs) File "/home/a19284/.qaihm/models/openai_clip/v1/openai_CLIP_git/clip/model.py", line 224, in forward x = self.conv1(x) # shape = [, width, grid, grid] File "/home/a19284/mambaforge/envs/qai_hub/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(input, kwargs) File "/home/a19284/mambaforge/envs/qai_hub/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 463, in forward return self._conv_forward(input, self.weight, self.bias) File "/home/a19284/mambaforge/envs/qai_hub/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 459, in _conv_forward return F.conv2d(input, weight, bias, self.stride, RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument weight in method wrapper___slow_conv2d_forward)

mestrona-3 commented 5 months ago

Closing this as it has been fixed in the latest release (yesterday) or Model Zoo.