That is because torch vision.transforms.Resize interpolates with bilinear everything which is wrong when used for segmentation labels - you just cannot mix them. Overall, torchvision.transforms is designed for classification only and cannot be used to images and masks together, unless you write two separate branches of augmentations.
Describe the bug
https://huggingface.co/docs/datasets/en/semantic_segmentation shows wrong example with torchvision transforms. Specifically, as one can see in screenshot below, the object boundaries have weird colors.
Original example with
albumentationsis correctThat is because
torch vision.transforms.Resizeinterpolates with bilinear everything which is wrong when used for segmentation labels - you just cannot mix them. Overall,torchvision.transformsis designed for classification only and cannot be used to images and masks together, unless you write two separate branches of augmentations.The correct way would be to use
v2version of transforms and convert the segmentation labels to https://pytorch.org/vision/main/generated/torchvision.tv_tensors.Mask.html#torchvision.tv_tensors.Mask objectSteps to reproduce the bug
Go to the website.
https://huggingface.co/docs/datasets/en/semantic_segmentation
Expected behavior
Results, similar to
albumentation. Or remove the torch vision part altogether. Or usekorniainstead.Environment info
Irrelevant