I am getting this error when trying to run train.py - any idea what this means or how to fix it?
Start training...
Traceback (most recent call last):
File "train.py", line 144, in main
train_one_epoch(writer, device, model, optimizer, scaler, train_dataloader, epoch, cfg)
File "train.py", line 238, in train_one_epoch
for x, y, metadata in metric_logger.log_every(data_loader, cfg.trainer.print_freq):
File "C:\Users\Admin\Documents\Apps\test\u-2-net-portrait\lib\utils\logger.py", line 184, in log_every
for obj in iterable:
File "C:\Users\Admin\Documents\Apps\test\u-2-net-portrait\lib\data\dataloader.py", line 14, in iter
for batch in self.loader:
File "C:\Users\Admin\anaconda3\envs\mypy36env\lib\site-packages\torch\utils\data\dataloader.py", line 521, in next
data = self._next_data()
File "C:\Users\Admin\anaconda3\envs\mypy36env\lib\site-packages\torch\utils\data\dataloader.py", line 560, in _next_data
index = self._next_index() # may raise StopIteration
File "C:\Users\Admin\anaconda3\envs\mypy36env\lib\site-packages\torch\utils\data\dataloader.py", line 512, in _next_index
return next(self._sampler_iter) # may raise StopIteration
File "C:\Users\Admin\anaconda3\envs\mypy36env\lib\site-packages\torch\utils\data\sampler.py", line 229, in iter
for idx in self.sampler:
File "C:\Users\Admin\anaconda3\envs\mypy36env\lib\site-packages\torch\utils\data\sampler.py", line 189, in iter
rand_tensor = torch.multinomial(self.weights, self.num_samples, self.replacement, generator=self.generator)
RuntimeError: invalid multinomial distribution (sum of probabilities <= 0)
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
Hello,
I am getting this error when trying to run train.py - any idea what this means or how to fix it?
Start training... Traceback (most recent call last): File "train.py", line 144, in main train_one_epoch(writer, device, model, optimizer, scaler, train_dataloader, epoch, cfg) File "train.py", line 238, in train_one_epoch for x, y, metadata in metric_logger.log_every(data_loader, cfg.trainer.print_freq): File "C:\Users\Admin\Documents\Apps\test\u-2-net-portrait\lib\utils\logger.py", line 184, in log_every for obj in iterable: File "C:\Users\Admin\Documents\Apps\test\u-2-net-portrait\lib\data\dataloader.py", line 14, in iter for batch in self.loader: File "C:\Users\Admin\anaconda3\envs\mypy36env\lib\site-packages\torch\utils\data\dataloader.py", line 521, in next data = self._next_data() File "C:\Users\Admin\anaconda3\envs\mypy36env\lib\site-packages\torch\utils\data\dataloader.py", line 560, in _next_data index = self._next_index() # may raise StopIteration File "C:\Users\Admin\anaconda3\envs\mypy36env\lib\site-packages\torch\utils\data\dataloader.py", line 512, in _next_index return next(self._sampler_iter) # may raise StopIteration File "C:\Users\Admin\anaconda3\envs\mypy36env\lib\site-packages\torch\utils\data\sampler.py", line 229, in iter for idx in self.sampler: File "C:\Users\Admin\anaconda3\envs\mypy36env\lib\site-packages\torch\utils\data\sampler.py", line 189, in iter rand_tensor = torch.multinomial(self.weights, self.num_samples, self.replacement, generator=self.generator) RuntimeError: invalid multinomial distribution (sum of probabilities <= 0)
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.