search

huawei-noah / trustworthyAI

Trustworthy AI related projects
Apache License 2.0
958 stars 216 forks source link

Tensors in different devices #135

Closed cloudy-sfu closed 9 months ago

cloudy-sfu commented 11 months ago

There is a problem in gcastle. After debugging, I believe it is caused by pytorch, so I proposed an issue in pytorch repository.

https://github.com/pytorch/pytorch/issues/111573

I suggest gcastle to join and track this issue also, since the failure only occurs when I use gcastle. The reason is still unknown from my view.

RishabhSamra commented 9 months ago

I am also facing the similar issue while running the gradient-based algorithm on GPU. @huawei-noah-admin , @cloudy-sfu , any suggestions how we can rectify this issue of tensors on different device when I specify device_type= GPU and also device_ids =1 ?

cloudy-sfu commented 9 months ago

I am also facing the similar issue while running the gradient-based algorithm on GPU. @huawei-noah-admin , @cloudy-sfu , any suggestions how we can rectify this issue of tensors on different device when I specify device_type= GPU and also device_ids =1 ?

If you are using torch as the backend:

I patched the "adam" file in torch. You can find the patch file at

https://github.com/cloudy-sfu/Amazon-molecule-literature/blob/16b3382abb14c5dfd06d2be1554992e3596c9cc1/method_gcastle/patch/adam.py

Unfortunately, I forgot where I modify, so please compare it with the source code of "torch", corresponding to the following official version, on your own.

torch-2.1.0+cu118-cp311-cp311-linux_x86_64.whl sha256=bcb17e2de6ca634d326203694d0bfb552587335e536c1917be3f28c5664b5506

The script to install the patch files:

https://github.com/cloudy-sfu/Amazon-molecule-literature/blob/main/method_gcastle/install.py