ValueError: index can't contain negative values

[Phyrokar]

🐛 Describe the bug

hello I wanted to train my custom dataset

from super_gradients.training.dataloaders import coco2017_train_yolo_nas, coco2017_val_yolo_nas

train_dataset_params = {
    'data_dir': '/home/phyrokar/beachpics',
    'subdir': 'images',
    'json_file': 'train_instances.json', 
    'input_dim': [736, 1280],
    'ignore_empty_annotations': True
}
train_data = coco2017_train_yolo_nas(
    dataset_params=train_dataset_params,
    dataloader_params={'batch_size':16}
)

# create val dataloader
val_dataset_params = {
    'data_dir': '/home/phyrokar/beachpics_val',
    'subdir': 'images',
    'json_file': 'val_instances.json', 
    'input_dim': [736, 1280],
    'ignore_empty_annotations': True
}    
val_data = coco2017_val_yolo_nas(
    dataset_params=val_dataset_params,
    dataloader_params={'batch_size':16}
)

and Im running over the following error

ValueError                                Traceback (most recent call last)
Cell In[6], line 1
----> 1 val_data.dataset.plot()
      3 # from data_gradients.managers.detection_manager import DetectionAnalysisManager
      4 
      5 # # train_data = ...  # Your dataset iterable (torch dataset[/dataloader/...](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f677269702f536f6d745f547261696e696e67222c22636f6e66696746696c65223a7b22246d6964223a312c22667350617468223a222f686f6d652f677269702f536f6d745f547261696e696e672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2265787465726e616c223a2266696c653a2f2f2f686f6d652f677269702f536f6d745f547261696e696e672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2270617468223a222f686f6d652f677269702f536f6d745f547261696e696e672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a2266696c65227d7d.vscode-resource.vscode-cdn.net/dataloader/...))
   (...)
     15 
     16 # analyzer.run()

File [/usr/local/lib/python3.8/dist-packages/super_gradients/training/datasets/detection_datasets/detection_dataset.py:528](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f677269702f536f6d745f547261696e696e67222c22636f6e66696746696c65223a7b22246d6964223a312c22667350617468223a222f686f6d652f677269702f536f6d745f547261696e696e672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2265787465726e616c223a2266696c653a2f2f2f686f6d652f677269702f536f6d745f547261696e696e672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2270617468223a222f686f6d652f677269702f536f6d745f547261696e696e672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a2266696c65227d7d.vscode-resource.vscode-cdn.net/usr/local/lib/python3.8/dist-packages/super_gradients/training/datasets/detection_datasets/detection_dataset.py:528), in DetectionDataset.plot(self, max_samples_per_plot, n_plots, plot_transformed_data)
    525 index = img_i + plot_i * 16
    527 if plot_transformed_data:
--> 528     image, targets, *_ = self[img_i + plot_i * 16]
    529     image = image.transpose(1, 2, 0).astype(np.int32)
    530 else:

File [/usr/local/lib/python3.8/dist-packages/super_gradients/training/datasets/detection_datasets/detection_dataset.py:426](https://vscode-remote+dev-002dcontainer-002b7b22686f737450617468223a222f686f6d652f677269702f536f6d745f547261696e696e67222c22636f6e66696746696c65223a7b22246d6964223a312c22667350617468223a222f686f6d652f677269702f536f6d745f547261696e696e672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2265787465726e616c223a2266696c653a2f2f2f686f6d652f677269702f536f6d745f547261696e696e672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c2270617468223a222f686f6d652f677269702f536f6d745f547261696e696e672f2e646576636f6e7461696e65722f646576636f6e7461696e65722e6a736f6e222c22736368656d65223a2266696c65227d7d.vscode-resource.vscode-cdn.net/usr/local/lib/python3.8/dist-packages/super_gradients/training/datasets/detection_datasets/detection_dataset.py:426), in DetectionDataset.__getitem__(self, index)
    419 def __getitem__(self, index: int) -> Tuple:
    420     """Get the sample post transforms at a specific index of the dataset.
    421     The output of this function will be collated to form batches.
    422 
    423     :param index:   Index refers to the index of the sample in the dataset, AFTER filtering (if relevant). 0<=index<=len(dataset)-1
    424     :return:        Sample, i.e. a dictionary including at least "image" and "target"
...
    516 # Converting the array with `tolist` seems to improve performance
    517 # when iterating and indexing the result (see usage in `pad`)
    518 return np.broadcast_to(x, (ndim, 2)).tolist()

ValueError: index can't contain negative values

Can someone help me with this?

Versions

super-gradients 3.2.1

[BloodAxe]

It looks like you are having issues inside the data gradients. What version of DG you are using there? FYI @Louis-Dupont

[Phyrokar]

@BloodAxe Thanks for the quick reply I use super-gradients 3.2.1

[BloodAxe]

We have added check for classes validity in detection dataset in 3.3.1 release of SG. I hope now you should be getting a more meaningful exception with traces where it is happening and what values are wrong. Can you give it a try and let us know whether this help?