The system cannot find the path specified: 'facemotortrain'

[tehreemnaqvi]

Hi miladmozafari, I am trying to implement your SpykeTorch code.I have some issues. First, in your ipynb file kept face and motorbicyle images but i got this error:

ValueError: Unknown resampling filter (64). Use Image.NEAREST (0), Image.LANCZOS (1), Image.BILINEAR (2), Image.BICUBIC (3), Image.BOX (4) or Image.HAMMING (5) Here i am using GaborFilter, Should i use DoG filters instead? Or i need to chanege some parametes?

Second: i am trying you code "MozafariShallow" when i tried to run your code i got this error: Waiting for your response. Thanks

[miladmozafari]

Hi, In the ipynb file a subset of ETH-80 is used. I cannot understand that you kept face and motorbike. Also I have never seen such an error. Can you put your code here?

For your second question, the error means that you do not have the folder containing face and motorbike images. You should modify the the code and put the address to the dataset folder on your disk.

[tehreemnaqvi]

Actually I want to apply your ipynb code on face and motorbike instead of cups and dogs. Second issue I solved. Now I am apply faces images on your reward based STDP.Here both training and testing images have size (510,337). I got this error: RuntimeError: stack expects each tensor to be equal size, but got [15, 4, 55, 83] at entry 0 and [15, 4, 56, 85] at entry 1

This is your code i am following: import os os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]="0"

import torch import torch.nn as nn from torch.utils.data import DataLoader from torchvision.datasets import ImageFolder import numpy as np from SpykeTorch import snn from SpykeTorch import functional as sf from SpykeTorch import visualization as vis from SpykeTorch import utils from torchvision import transforms

class Mozafari2018(nn.Module): def init(self, input_channels, features_per_class, number_of_classes, s2_kernel_size, threshold, stdp_lr, anti_stdp_lr, dropout = 0.): super(Mozafari2018, self).init() self.features_per_class = features_per_class self.number_of_classes = number_of_classes self.number_of_features = features_per_class number_of_classes self.kernel_size = s2_kernel_size self.threshold = threshold self.stdp_lr = stdp_lr self.anti_stdp_lr = anti_stdp_lr self.dropout = torch.ones(self.number_of_features) dropout self.to_be_dropped = torch.bernoulli(self.dropout).nonzero()

    self.s2 = snn.Convolution(input_channels, self.number_of_features, self.kernel_size, 0.8, 0.05)
    self.stdp = snn.STDP(self.s2, stdp_lr)
    self.anti_stdp = snn.STDP(self.s2, anti_stdp_lr)
    self.decision_map = []
    for i in range(number_of_classes):
        self.decision_map.extend([i]*features_per_class)

    self.ctx = {"input_spikes":None, "potentials":None, "output_spikes":None, "winners":None}

def forward(self, input):
    input = input.float()
    pot = self.s2(input)

    if self.training and self.dropout[0] > 0:
        sf.feature_inhibition_(pot, self.to_be_dropped)

    spk, pot = sf.fire(pot, self.threshold, True)
    winners = sf.get_k_winners(pot, 1, 0, spk)
    output = -1
    if len(winners) != 0:
        output = self.decision_map[winners[0][0]]

    if self.training:
        self.ctx["input_spikes"] = input
        self.ctx["potentials"] = pot
        self.ctx["output_spikes"] = spk
        self.ctx["winners"] = winners
    else:
        self.ctx["input_spikes"] = None
        self.ctx["potentials"] = None
        self.ctx["output_spikes"] = None
        self.ctx["winners"] = None

    return output

def update_dropout(self):
    self.to_be_dropped = torch.bernoulli(self.dropout).nonzero()

def update_learning_rates(self, stdp_ap, stdp_an, anti_stdp_ap, anti_stdp_an):
    self.stdp.update_all_learning_rate(stdp_ap, stdp_an)
    self.anti_stdp.update_all_learning_rate(anti_stdp_an, anti_stdp_ap)

def reward(self):
    self.stdp(self.ctx["input_spikes"], self.ctx["potentials"], self.ctx["output_spikes"], self.ctx["winners"])

def punish(self):
    self.anti_stdp(self.ctx["input_spikes"], self.ctx["potentials"], self.ctx["output_spikes"], self.ctx["winners"])

class S1C1Transform: def init(self, filter, pooling_size, pooling_stride, lateral_inhibition = None, timesteps = 15, feature_wise_inhibition=True): self.grayscale = transforms.Grayscale() self.to_tensor = transforms.ToTensor() self.filter = filter self.pooling_size = pooling_size self.pooling_stride = pooling_stride self.lateral_inhibition = lateral_inhibition self.temporal_transform = utils.Intensity2Latency(timesteps) self.feature_wise_inhibition = feature_wise_inhibition

def __call__(self, image):
    image = self.to_tensor(self.grayscale(image))
    image.unsqueeze_(0)
    image = self.filter(image)
    image = sf.pooling(image, self.pooling_size, self.pooling_stride, padding=self.pooling_size//2)
    if self.lateral_inhibition is not None:
        image = self.lateral_inhibition(image)
    temporal_image = self.temporal_transform(image)
    temporal_image = sf.pointwise_inhibition(temporal_image)
    return temporal_image.sign().byte()

kernels = [ utils.GaborKernel(5, 45+22.5), utils.GaborKernel(5, 90+22.5), utils.GaborKernel(5, 135+22.5), utils.GaborKernel(5, 180+22.5)]

filter = utils.Filter(kernels, use_abs = True) lateral_inhibition = utils.LateralIntencityInhibition([0.15, 0.12, 0.1, 0.07, 0.05])

use_cuda = True

C:\Users\Tehreem\Desktop\CV LAB DATA\SNN data\SDNN_STDP\DataSet\Caltech\TestingSet\FaceMotor

data_root = '/Users/Tehreem/Desktop/CV LAB DATA/SNN data/SDNN_STDP/DataSet/Caltech/TrainingSet/' root ='/Users/Tehreem/Desktop/CV LAB DATA/SNN data/SDNN_STDP/DataSet/Caltech/TestingSet/'

s1c1 = S1C1Transform(filter, 7, 6, lateral_inhibition) trainsetfolder = utils.CacheDataset(ImageFolder(data_root, s1c1)) testsetfolder = utils.CacheDataset(ImageFolder(root, s1c1)) mozafari = Mozafari2018(4, 10, 2, (17,17), 60, (0.005, -0.0025), (-0.005, 0.0005), 0.5) trainset = DataLoader(trainsetfolder, batch_size = len(trainsetfolder), shuffle = True) testset = DataLoader(testsetfolder, batch_size = len(testsetfolder), shuffle = True) max_epoch = 400

if use_cuda: mozafari.cuda()

initial adaptive learning rates

apr = mozafari.stdp_lr[0] anr = mozafari.stdp_lr[1] app = mozafari.anti_stdp_lr[1] anp = mozafari.anti_stdp_lr[0]

adaptive_min = 0.2 adaptive_int = 0.8 apr_adapt = ((1.0 - 1.0 / mozafari.number_of_classes) adaptive_int + adaptive_min) apr anr_adapt = ((1.0 - 1.0 / mozafari.number_of_classes) adaptive_int + adaptive_min) anr app_adapt = ((1.0 / mozafari.number_of_classes) adaptive_int + adaptive_min) app anp_adapt = ((1.0 / mozafari.number_of_classes) adaptive_int + adaptive_min) anp

perf

best_train = np.array([0,0,0,0]) # correct, wrong, silence, epoch best_test = np.array([0,0,0,0]) # correct, wrong, silence, epoch

train one batch (here a batch contains all data so it is an epoch)

def train(data, target, network): network.train() perf = np.array([0,0,0]) # correct, wrong, silence network.update_dropout() for i in range(len(data)): data_in = data[i] target_in = target[i] if use_cuda: data_in = data_in.cuda() target_in = target_in.cuda() d = network(data_in) if d != -1: if d == target_in: perf[0]+=1 network.reward() else: perf[1]+=1 network.punish() else: perf[2]+=1 return perf/len(data)

test one batch (here a batch contains all data so it is an epoch)

def test(data, target, network): network.eval() perf = np.array([0,0,0]) # correct, wrong, silence for i in range(len(data)): data_in = data[i] target_in = target[i] if use_cuda: data_in = data_in.cuda() target_in = target_in.cuda() d = network(data_in) if d != -1: if d == target_in: perf[0]+=1 else: perf[1]+=1 else: perf[2]+=1 return perf/len(data)

for epoch in range(max_epoch): print("Epoch #:", epoch) for data, target in trainset: perf_train = train(data, target, mozafari) if best_train[0] <= perf_train[0]: best_train = np.append(perf_train, epoch) print("Current Train:", perf_train) print(" Best Train:", best_train) for data_test, target_test in testset: perf_test = test(data_test, target_test, mozafari) if best_test[0] <= perf_test[0]: best_test = np.append(perf_test, epoch) torch.save(mozafari.state_dict(), "saved.net") print(" Current Test:", perf_test) print(" Best Test:", best_test)

#update adaptive learning rates
apr_adapt = apr * (perf_train[1] * adaptive_int + adaptive_min)
anr_adapt = anr * (perf_train[1] * adaptive_int + adaptive_min)
app_adapt = app * (perf_train[0] * adaptive_int + adaptive_min)
anp_adapt = anp * (perf_train[0] * adaptive_int + adaptive_min)
mozafari.update_learning_rates(apr_adapt, anr_adapt, app_adapt, anp_adapt)

Features

feature = torch.tensor([ [ [1] ] ]).float() if use_cuda: feature = feature.cuda()

cstride = (1,1)

S1 Features

if use_cuda: feature,cstride = vis.get_deep_feature(feature, cstride, (filter.max_window_size, filter.max_window_size), (1,1), filter.kernels.cuda()) else: feature,cstride = vis.get_deep_feature(feature, cstride, (filter.max_window_size, filter.max_window_size), (1,1), filter.kernels)

C1 Features

feature,cstride = vis.get_deep_feature(feature, cstride, (s1c1.pooling_size, s1c1.pooling_size), (s1c1.pooling_stride, s1c1.pooling_stride))

S2 Features

feature,cstride = vis.get_deep_feature(feature, cstride, mozafari.kernel_size, (1,1), mozafari.s2.weight)

for i in range(mozafari.number_of_features): vis.plot_tensor_in_image('features2'+str(i).zfill(4)+'.png',feature[i])

[tehreemnaqvi]

Did you use same image size for training and testing images Caltech (Facemotor) dataset? I am using Facemotor images with same sizes for training and testing.

[miladmozafari]

Yes, the problem is that your images are in different sizes. In our paper, I resized all the images to a fixed size.

[tehreemnaqvi]

Thank you for your response.

On Sun, May 3, 2020, 12:56 AM miladmozafari notifications@github.com wrote:

Yes, the problem is that your images are in different sizes. In our paper, I resized all the images to a fixed size.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/miladmozafari/SpykeTorch/issues/4#issuecomment-622974863, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJFFXM2GWF22EQPQN7WLQDDRPQ7CHANCNFSM4MW5LE5A .