es-imagenet-master

Latest News: Now you can use SpikingJelly to process ES-ImageNet!

API reference: ES-imageNet API

checkpoints download: google driver

code for generating data set ES-ImageNet with corresponding training code

dataset generator

• some codes of ODG algorithm

• The variables to be modified include datapath (data storage path after transformation, which needs to be created before transformation) and root_Path (root directory of training set before transformation)

  file name	function
  traconvert.py	converting training set of ISLVRC 2012 into event stream using ODG
  trainlabel_dir.txt	It stores the corresponding relationship between the class name and label of the original Imagenet file
  trainlabel.txt	It is generated during transformation and stores the label of training set
  valconvert.py	Transformation code for test set.
  valorigin.txt	Original test label of ImageNet-1K. Put it in the same folder with valconvert.py if you need.
  vallabel.txt	It is generated during transformation and stores the label of training set.

dataset usage

• codes are in ./datasets

• some traing examples are provided for ES-imagenet in ./example An example code for easily using this dataset based on Pytorch

  from __future__ import print_function
  import sys
  sys.path.append("..")
  from datasets.es_imagenet_new import ESImagenet_Dataset
  import torch.nn as nn
  import torch
  
  data_path = None #TODO:modify 
  train_dataset = ESImagenet_Dataset(mode='train',data_set_path=data_path)
  test_dataset = ESImagenet_Dataset(mode='test',data_set_path=data_path)
  
  train_sampler = torch.utils.data.distributed.DistributedSampler(train_dataset)
  test_sampler  = torch.utils.data.distributed.DistributedSampler(test_dataset)
  train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=False, num_workers=1,pin_memory=True,drop_last=True,sampler=train_sampler)
  test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=batch_size, shuffle=False, num_workers=1,pin_memory=True)
  
  for batch_idx, (inputs, targets) in enumerate(train_loader)
  pass
  # input = [batchsize,time,channel,width,height]
  
  for batch_idx, (inputs, targets) in enumerate(test_loader):
  pass
  # input = [batchsize,time,channel,width,height]

  training example and benchmarks

  Requirements

• Python >= 3.5
• Pytorch >= 1.7
• CUDA >=10.0

• TenosrBoradX(optional)

  Train the baseline models

  
  $ cd example
  
  $ CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 example_ES_res18.py #LIAF/LIF-ResNet-18
  $ CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 example_ES_res34.py #LIAF/LIF-ResNet-34
  $ CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 compare_ES_3DCNN34.py #3DCNN-ResNet-34
  $ CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 compare_ES_3DCNN18.py #3DCNN-ResNet-18
  $ CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 compare_ES_2DCNN34.py #2DCNN-ResNet-34 
  $ CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 compare_ES_2DCNN18.py #2DCNN-ResNet-18
  $ CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 compare_CONVLSTM.py #ConvLSTM (no used in paper)
  $ CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4 example_ES_res50.py #LIAF/LIF-ResNet-50 (no used in paper)

note: To select LIF mode, change the config files under /LIAFnet : self.actFun= torch.nn.LeakyReLU(0.2, inplace=False) #nexttest:selu to self.actFun= LIAF.LIFactFun.apply

baseline / Benchmark

Note: model (a), (b) and (c) are stored in ./pretrained_model Some problems related to model loading can be referred to the issue. If you want to test 2D-CNN with reconstructed gray set, you can use this notebook

Download

• The datasets ES-ImageNet (100GB) for this study can be download in the Tsinghua Cloud1 or Openl

• The converted event-frame version (40GB, events are organized as 8 event-frames instead of lists) can be found in Tsinghua Cloud2, IT MAY BE BUGGY, STILL NEED TO BE CHECKED

• If you only need the validation set, you can download it in Tsinghua Cloud3 separately

Citation

If you find this code useful in your research, please consider citing and here is an example BibTeX entry:

@ARTICLE{ES_ImageNet2021,
AUTHOR={Lin, Yihan and Ding, Wei and Qiang, Shaohua and Deng, Lei and Li, Guoqi},   
TITLE={ES-ImageNet: A Million Event-Stream Classification Dataset for Spiking Neural Networks},      
JOURNAL={Frontiers in Neuroscience},      
VOLUME={15},      
PAGES={1546},     
YEAR={2021},          
URL={https://www.frontiersin.org/article/10.3389/fnins.2021.726582},        
DOI={10.3389/fnins.2021.726582},      
ISSN={1662-453X},   
}