What are the expected tensor types for params?

[jodusan]

In the following function type of the middle two params are specified, but first two and last two arent. Also what device is compatible? It says CUDA in readme but when you use cuda tensors, the call to function compute_wer fails because it has .float() and cuda tensor doesnt have that casting. So are CPU tensors expected?

def levenshtein_distance(
        hypotheses,             # type: torch.Tensor
        references,             # type: torch.Tensor
        hypothesis_lengths,     # type: torch.IntTensor
        references_lengths,     # type: torch.IntTensor
        blank,                  # type: torch.Tensor
        separator               # type: torch.Tensor
):

[jodusan]

Okay so when I put CPU tensors it complains that source, target, source_length, target_length, blank, space all must be CUDA tensor

[jodusan]

I have managed to run it by converting first two and last two to Long type. But now I have issue on backward pass (i don't have any issue if I use model.eval()).

on loss.backward() 
...
  File "/opt/miniconda3/envs/pytorchasr/lib/python3.7/site-packages/torch/tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/opt/miniconda3/envs/pytorchasr/lib/python3.7/site-packages/torch/autograd/__init__.py", line 93, in backward
    grad_tensors = _make_grads(tensors, grad_tensors)
  File "/opt/miniconda3/envs/pytorchasr/lib/python3.7/site-packages/torch/autograd/__init__.py", line 35, in _make_grads
    new_grads.append(torch.ones_like(out, memory_format=torch.preserve_format))
RuntimeError: CUDA error: invalid device function

How do I disable the backward pass?

Thanks

[fightingyuman]

@dulex123 Hello, have you solve this issue? I'm new of nlp, can you show me how to transfer strings to tensors?

[ShunLu91]

I met the same problem. Have you solved this problem? I would really appreciate if anyone could give me some help.

Traceback (most recent call last):
  File "train_supernet_spos.py", line 55, in <module>
    main()
  File "train_supernet_spos.py", line 51, in main
    trainer.train(model, epochs=args.epochs, lr=args.lr, reset=args.reset, model_name=args.exp_name)
  File "/home/lushun/code/nas-lib-map/source_repos/nb-asr-main/nasbench_asr/training/torch/trainer_spos.py", line 169, in train
    per = self.decode(logits, logits_len, val_input)
  File "/home/lushun/code/nas-lib-map/source_repos/nb-asr-main/nasbench_asr/training/torch/trainer_spos.py", line 271, in decode
    per = ed.compute_wer(top_beams, targets, top_beams_len, targets_len, blank, sep)
  File "/home/lushun/anaconda3/envs/pt1.7.0/lib/python3.7/site-packages/torch_edit_distance/__init__.py", line 97, in compute_wer
    data = levenshtein_distance(hs, rs, hn, rn, blank, space).float()
RuntimeError: CUDA error: invalid device function