Trouble Running Perf.py in Benchmark

hpcaitech / FastFold

Optimizing AlphaFold Training and Inference on GPU Clusters

Apache License 2.0

556 stars 84 forks source link

Trouble Running Perf.py in Benchmark #178

Closed chuhan-ouyang closed 7 months ago

chuhan-ouyang commented 1 year ago

Hi,

After the fix to the perf.py CLI issue, there is still a problem in running the file. It seems to be a mismatch between the size of the parameters between the layers.

Gy-Lu commented 1 year ago

Hi, but I can not reproduce this problem. If you want to examine the shape, just print the tensor shape before this line:

But I think it might be something wrong with your environment. Here is some discussion about this problem which you can refer: https://discuss.pytorch.org/t/runtimeerror-cuda-error-cublas-status-invalid-value-when-calling-cublassgemm-handle-opa-opb-m-n-k-alpha-a-lda-b-ldb-beta-c-ldc/124544/18

chuhan-ouyang commented 7 months ago

Fixed by running using CUDA 11.6 (CUDA 11.3 should also work).