jordiclive / ControlPrefixes

Apache License 2.0
90 stars 11 forks source link

I cannot reproduce the finetuning procedure by following README #15

Open daehuikim opened 7 months ago

daehuikim commented 7 months ago

Hello, I am trying to test my data in Control Prefix manner. While I am testing, I always encounter error like below.

Validation sanity check: 0it [00:00, ?it/s]
Validation sanity check:   0%|          | 0/4 [00:00<?, ?it/s]Traceback (most recent call last):
  File "cptest/src/datatotext/finetune.py", line 932, in <module>
    model = main(args)
   ...
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)`

What Can I do for solving this problem? there is nothing I modified in files in src/ directory. Thanks for watching this!