search

nanoporetech / bonito

A PyTorch Basecaller for Oxford Nanopore Reads
https://nanoporetech.com/
Other
394 stars 121 forks source link

RuntimeError: target_lengths must be of size batch_size #197

Open gao9842 opened 2 years ago

gao9842 commented 2 years ago

The following error was reported when I ran CNN models with 4 gpus under bonito 0.4.0, but it works fine on bonito 0.2.3. I'd appreciate any suggestions!

[loading data] [validation set not found: splitting training set] [loading model] [0/1672047]: 0%| | [00:00] Traceback (most recent call last): File "/mnt/gvol/anaconda3/envs/bonito/bin/bonito", line 33, in sys.exit(load_entry_point('ont-bonito', 'console_scripts', 'bonito')()) File "/mnt/gvol/bonito/bonito/init.py", line 34, in main args.func(args) File "/mnt/gvol/bonito/bonito/cli/train.py", line 104, in main train_loss, duration = train( File "/mnt/gvol/bonito/bonito/training.py", line 154, in train losses = criterion(scores, targets.to(device), lengths.to(device)) File "/mnt/gvol/bonito/bonito/training.py", line 121, in ctc_label_smoothing_loss loss = ctc_loss(log_probs.to(torch.float32), targets, log_probs_lengths, lengths, reduction='mean') File "/mnt/gvol/gzy/anaconda3/envs/bonito/lib/python3.8/site-packages/torch/nn/functional.py", line 2304, in ctc_loss return torch.ctc_loss( RuntimeError: target_lengths must be of size batch_size

Thank you in advance!

iiSeymour commented 2 years ago

Hi @gao9842

Can you try with v0.5 and a single GPU?