获取事件数据使用自定义transform参数报错

[2ephyrus]

Read before creating a new issue

• Users who want to use SpikingJelly should first be familiar with the usage of PyTorch.
• If you do not know much about PyTorch, we recommend that the user can learn the basic tutorials of PyTorch.
• Do not ask for help with the basic conception of PyTorch/Machine Learning but not related to SpikingJelly. For these questions, please refer to Google or PyTorch Forums.

For faster response

You can @ the corresponding developers for your issue. Here is the division:

Features	Developers
Neurons and Surrogate Functions	fangwei123456 Yanqi-Chen
CUDA Acceleration	fangwei123456 Yanqi-Chen
Reinforcement Learning	lucifer2859
ANN to SNN Conversion	DingJianhao Lyu6PosHao
Biological Learning (e.g., STDP)	AllenYolk
Others	Grasshlw lucifer2859 AllenYolk Lyu6PosHao DingJianhao Yanqi-Chen fangwei123456

We are glad to add new developers who are volunteering to help solve issues to the above table.

Issue type

• [ ] Bug Report
• [ ] Feature Request
• [x] Help wanted
• [ ] Other

SpikingJelly version

0.0.0.0.2

Description

报错如下： Traceback (most recent call last): File "E:\GitCode\MLF-DSResNet\train_for_Caltech101.py", line 126, in train_loader1, test_loader1, start_epoch = data_model_load(args, model, kwargs) File "E:\GitCode\MLF-DSResNet\train_for_Caltech101.py", line 17, in data_model_load train_dataset, test_dataset = NCaltech_preprocess.splitdataset() File "E:\GitCode\MLF-DSResNet\NCaltech_preprocess.py", line 19, in splitdataset return sjds.split_to_train_test_set(train_ratio = 0.9, origin_dataset = event_set, num_classes = 101, random_split = False) File "E:\anaconda3\lib\site-packages\spikingjelly\datasets__init__.py", line 458, in split_to_train_test_set for i, item in enumerate(tqdm.tqdm(origin_dataset)): File "E:\anaconda3\lib\site-packages\tqdm\std.py", line 1195, in iter for obj in iterable: File "E:\anaconda3\lib\site-packages\torchvision\datasets\folder.py", line 231, in getitem sample = self.transform(sample) File "E:\anaconda3\lib\site-packages\torchvision\transforms\transforms.py", line 95, in call img = t(img) File "E:\anaconda3\lib\site-packages\torchvision\transforms\transforms.py", line 234, in call return F.to_pil_image(pic, self.mode) File "E:\anaconda3\lib\site-packages\torchvision\transforms\functional.py", line 278, in to_pil_image raise ValueError(f"pic should be 2/3 dimensional. Got {pic.ndim} dimensions.") ValueError: pic should be 2/3 dimensional. Got 4 dimensions.

from spikingjelly.datasets.n_caltech101 import NCaltech101
import spikingjelly.datasets as sjds
import torchvision.transforms as transforms
# 超参数设置
TS = 5  # 采用全段时间进行分割，这样做，dt会十分长
size = [42, 42]
# ds = [4.29, 3.04] # size = [2, 42, 42]
# 定义 transform
transform = transforms.Compose([
    transforms.ToPILImage(),  # 将 numpy 数组转换为 PIL 图像
    transforms.Resize(size=(42, 42)), # 降采样到指定尺寸
    # transforms.ToTensor(), # 将 PIL 图像或 NumPy ndarray 转换为 FloatTensor，并缩放像素的强度值到 [0., 1.]
])
# 解压 + 降采样
root_dir = 'E:\GitCode\MLF-DSResNet/data/N_Caltech/source'
event_set = NCaltech101(root=root_dir, data_type='frame', frames_number=TS, split_by='number', transform=transform)
# 划分训练集与测试集 random_split 置为 False，按照从前到后的顺序 划分训练集与测试集。 置为True，则使用随机种子进行分割。
def splitdataset():
    return sjds.split_to_train_test_set(train_ratio = 0.9, origin_dataset = event_set, num_classes = 101, random_split = False)
# ...

[2ephyrus]

使用for循环解决了上述问题，不知道处理的是否正确。代码如下：

from spikingjelly.datasets.n_caltech101 import NCaltech101
import spikingjelly.datasets as sjds
import torchvision.transforms as transforms
import cv2
import numpy as np
# 超参数设置
TS = 5  # 采用全段时间进行分割，这样做，dt会十分长
size = [42, 42]
# ds = [4.29, 3.04] # size = [2, 42, 42]
# 定义 transform
transform = transforms.Compose([
    transforms.ToPILImage(),  # 将 numpy 数组转换为 PIL 图像
    transforms.Resize(size=(42, 42)), # 降采样到指定尺寸
    # transforms.ToTensor(), # 将 PIL 图像或 NumPy ndarray 转换为 FloatTensor，并缩放像素的强度值到 [0., 1.]
])
# 解压
root_dir = 'E:\GitCode\MLF-DSResNet/data/N_Caltech/source'
event_set = NCaltech101(root=root_dir, data_type='frame', frames_number=TS, split_by='number')
# 降采样 由于维度为 4 不能直接使用resize 需要对每一帧数据单独降采样
ds_set = []
for i in range(len(event_set)):
    # x形状为[T, C, H, W]  ，y为标签值。
    x, y = event_set[i]
    xx = []
    T = len(x[:, 0, 0, 0])
    C = len(x[0, :, 0, 0])
    for j in range(T):
        for k in range(C):
            xx.append(cv2.resize(x[j, k, :, :], (42, 42)))
    xx = np.array(xx).reshape(T, C, 42, 42)
    ds_set.append((xx, y))
    print(i)
# 划分训练集与测试集 random_split 置为 False，按照从前到后的顺序 划分训练集与测试集。 置为True，则使用随机种子进行分割。
def splitdataset():
    return sjds.split_to_train_test_set(train_ratio = 0.9, origin_dataset = ds_set, num_classes = 101, random_split = False)

[fangwei123456]

可以这样，因为原始的torchvision中的图片处理方法并不支持数据多一个时间维度

[fangwei123456]

有空应该在框架里面加一些函数使得能兼容

[2ephyrus]

感谢作者的回复。针对固定时间步长duration这个参数的使用，希望框架能够提供一个新的参数使得提取出指定个数的帧数据。 即有些情况下无需用到所有的帧数据，如果全部提取出来所需要的时间过长。 另外，如果需要对这些数据进行降采样，对帧数据进行提取所占用的时间会影响整个降采样循环的速度，具体可见下述代码：

for i in range(len(event_set)):
    # x形状为[T, C, H, W]  ，y为标签值。
    x, y = event_set[i]

如果duration很小，会造成event_set[i]数据量过大，导致循环这里读取速度太慢。 针对DVS-CIFAR10数据集，duration使用300ms与10ms，上述循环的用时分别为20s与8min左右。