hpcaitech / FastFold

Optimizing AlphaFold Training and Inference on GPU Clusters
Apache License 2.0
556 stars 84 forks source link

Could not run demo.py #192

Open bj600800 opened 3 months ago

bj600800 commented 3 months ago

I could not run demo.py Report RuntimeError.

Here is the details, -- Process 0 terminated with the following error: Traceback (most recent call last): File "/home/u2600215/miniconda3/envs/fastfold/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap fn(i, args) File "/home/u2600215/FastFold-main/demo.py", line 111, in inference_model out = model(batch) File "/home/u2600215/miniconda3/envs/fastfold/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(input, kwargs) File "/home/u2600215/FastFold-main/fastfold/model/hub/alphafold.py", line 522, in forward outputs, m_1_prev, z_prev, x_prev = self.iteration( File "/home/u2600215/FastFold-main/fastfold/model/hub/alphafold.py", line 203, in iteration self.input_embedder( File "/home/u2600215/miniconda3/envs/fastfold/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, *kwargs) File "/home/u2600215/FastFold-main/fastfold/model/fastnn/ops.py", line 1307, in forward tf_emb_i = self.linear_tf_z_i(tf) File "/home/u2600215/miniconda3/envs/fastfold/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(input, kwargs) File "/home/u2600215/miniconda3/envs/fastfold/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)

Thanks for any kind help.

bj600800 commented 3 months ago

In case of any further issue like this one, here is the solution:

Pay attention to your cudatoolkit path, by "which nvcc", if it use the other version of cudatoolkit. USE THE CONDA INSTALLED ONE!