Open erkara opened 1 year ago
When I try to load my custum dataset using the tutorial you provided, I got the following error.
File /home/erkara/TorchStudio/python/lib/python3.10/ctypes/init.py, line 374, in init self._handle = _dlopen(self._name, mode) OSError: /home/erkara/TorchStudio/python/lib/python3.10/site-packages/torchaudio/lib/libtorchaudio.so: undefined symbol: _ZNK3c107SymBool10guard_boolEPKcl
It appears there is some sort of incompatibility issue going on here. It is actually addressed here
When I try the same thing in Windows, I got a different error in getattr raise AttributeError(name) AttributeError: n_frames
which point to PIL
I already have a generic Pytorch code that works on the same dataset without any error but I got these errors in torchstudio Any help?
I think one workaround is to use CustomDataset option, check the "Code" box. I used the following generic template to load the data
class MyDataset(Dataset): def init(self, data_dir, transform=None): super().init() self.data_dir = data_dir self.transform = transform self.classes = sorted(os.listdir(self.data_dir)) self.class_to_idx = {c: i for i, c in enumerate(self.classes)} self.samples = self._make_samples()
def _make_samples(self):
samples = []
for class_name in self.classes:
class_dir = os.path.join(self.data_dir, class_name)
for file_name in os.listdir(class_dir):
file_path = os.path.join(class_dir, file_name)
samples.append((file_path, self.class_to_idx[class_name]))
return samples
def __len__(self):
return len(self.samples)
def __getitem__(self, idx):
file_path, class_idx = self.samples[idx]
image = Image.open(file_path).convert("RGB")
if self.transform is not None:
image = self.transform(image)
return image, class_idx
If you use code and replace the do_tensor function with the following one, it should work while keeping all the features as-is.
def to_tensors(self, path:str):
tensors = []
if path.endswith('.jpg') or path.endswith('.jpeg') or path.endswith('.png') or path.endswith('.webp') or path.endswith('.tif') or path.endswith('.tiff'):
img=Image.open(path)
frames = 1
if hasattr( img, 'n_frames'):
frames = img.n_frames
for i in range(frames):
if img.mode=='1' or img.mode=='L' or img.mode=='P':
tensors.append(torch.from_numpy(np.array(img, dtype=np.uint8)))
else:
trans=torchvision.transforms.ToTensor()
tensors.append(trans(img))
if i<(frames-1):
img.seek(img.tell()+1)
if path.endswith('.mp3') or path.endswith('.wav') or path.endswith('.ogg') or path.endswith('.flac'):
waveform, sample_rate = torchaudio.load(path)
tensors.append(waveform)
if path.endswith('.npy') or path.endswith('.npz'):
arrays = np.load(path)
if type(arrays) == dict:
for array in arrays:
tensors.append(torch.from_numpy(arrays[array]))
else:
tensors.append(torch.from_numpy(arrays))
return tensors
@kiriri thanks for the fix, I'll push it into the next release
When I try to load my custum dataset using the tutorial you provided, I got the following error.
File /home/erkara/TorchStudio/python/lib/python3.10/ctypes/init.py, line 374, in init self._handle = _dlopen(self._name, mode) OSError: /home/erkara/TorchStudio/python/lib/python3.10/site-packages/torchaudio/lib/libtorchaudio.so: undefined symbol: _ZNK3c107SymBool10guard_boolEPKcl
It appears there is some sort of incompatibility issue going on here. It is actually addressed here
When I try the same thing in Windows, I got a different error in getattr raise AttributeError(name) AttributeError: n_frames
which point to PIL
I already have a generic Pytorch code that works on the same dataset without any error but I got these errors in torchstudio Any help?