search

Dumitrescu-Alexandru / TSignal

MIT License
6 stars 2 forks source link

RuntimeError: CUDA error: no kernel image is available for execution on the device #5

Open Rohit-Satyam opened 1 month ago

Rohit-Satyam commented 1 month ago

I have 606 protein sequences from uniprot. I tried running TSignal as follows and ran into the following issue @Dumitrescu-Alexandru 

python main.py --test_seqs uniprot_all_true_forSignalPep.fasta --test_mdl deployment_sep_pe_swa_extra_inpemb_on_gen_best_eval_only_dec.pth --tune_bert --train_only_decoder

Can you help?


Created binary file for test set at sp_data/uniprot_all_true_forSignalPep.bin.
Loading model from sp_data/deployment_sep_pe_swa_extra_inpemb_on_gen_best_eval_only_dec.pth
/home/subudhak/miniconda3/envs/tsignal/lib/python3.7/site-packages/torch/cuda/__init__.py:104: UserWarning: 
NVIDIA RTX A4500 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70 sm_75.
If you want to use the NVIDIA RTX A4500 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

  warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
0 number of seqs out of 60 tested
Traceback (most recent call last):
  File "main.py", line 235, in <module>
    test_seqs_w_pretrained_mdl(args.test_mdl, args.test_seqs, tune_bert=args.tune_bert, saliency_map_save_fn=args.saliency_map_save_fn,hook_layer=args.hook_layer, compute_saliency=args.compute_saliency)
  File "/data/phdProjects/project1/dl/annoSeqDl/tools/TSignal/train_scripts/cv_train_cs_predictors.py", line 1935, in test_seqs_w_pretrained_mdl
    hook_layer=hook_layer)
  File "/data/phdProjects/project1/dl/annoSeqDl/tools/TSignal/train_scripts/cv_train_cs_predictors.py", line 212, in greedy_decode
    memory_bfd = model.ProtBertBFD(input_ids=input_ids, attention_mask=attention_mask)[0]
  File "/home/subudhak/miniconda3/envs/tsignal/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/subudhak/miniconda3/envs/tsignal/lib/python3.7/site-packages/transformers/modeling_bert.py", line 806, in forward
    token_type_ids = torch.zeros(input_shape, dtype=torch.long, device=device)
RuntimeError: CUDA error: no kernel image is available for execution on the device
Rohit-Satyam commented 1 month ago

the following version resolved the issue:

mamba install pytorch=1.8.1 torchvision cudatoolkit=10.2 -c pytorch

But I don't understand why the output is printed to the screen and not saved to a file. Also it lacks the Sequence header information

TRUE: ################################UNKNOWN###############################
PRED: OOOOOMMMMMMMMMMMMMMMMMMMMIIIIIIIIIIIIIIIIIIMMMMMMMMMMMMMMMMMMMMOOOOOOOES
TYPE PROB:0.981691837310791; CS PROB:N/A (no SP predicted)

TRUE: ################################UNKNOWN###############################
PRED: SSSSSSSSSSSSSSSSSSSSSSSSSSSSOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOES
TYPE PROB:0.9999996423721313; CS PROB:0.9999140501022339
Dumitrescu-Alexandru commented 3 weeks ago

Thank you for the feedback! I have updated the code. Please pull the latest updates.

Results are now saved (by default in output.csv, output file can be specified through --output_file parameter now - see also the updated example_runs.txt). Results are printed only when using --verbose argument.