VGG on CIFAR

[prachikashikar]

Hi, for my experiment I need to train VGG on CIFAR. I believe I can do that by writting a new .ini file of a model may be. Could you please tell if I can do that or how can I do that?

I tried doing the followng thing but, I think it is not correct as training is not done properly.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;    (C) Copyright 2017 CEA LIST. All Rights Reserved.
;    Contributor(s): David BRIAND (david.briand@cea.fr)
;
;    This software is governed by the CeCILL-C license under French law and
;    abiding by the rules of distribution of free software.  You can  use,
;    modify and/ or redistribute the software under the terms of the CeCILL-C
;    license as circulated by CEA, CNRS and INRIA at the following URL
;    "http://www.cecill.info".
;
;    As a counterpart to the access to the source code and  rights to copy,
;    modify and redistribute granted by the license, users are provided only
;    with a limited warranty  and the software's author,  the holder of the
;    economic rights,  and the successive licensors  have only  limited
;    liability.
;
;    The fact that you are presently reading this means that you have had
;    knowledge of the CeCILL-C license and that you accept its terms.
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

; ./n2d2.sh "$N2D2_MODELS/VGG16.ini" -learn 200000000 -log 512000

; Resolution
$SIZE=224
; Margin for random cropping during learning
$MARGIN=32

; Training parameters
$LR=0.01
$WD=0.0005
$MOMENTUM=0.9
$BATCH_SIZE=16
$STEP_DECAY_EPOCHS=39
$STEP_DECAY_RATE=0.1
$EPOCHS=$(4 * ${STEP_DECAY_EPOCHS})

DefaultModel=Frame_CUDA

; Database
[database]
Type=CIFAR10_Database
RandomPartitioning=0
Learn=1.0

; Environment
[sp]
SizeX=${SIZE}
SizeY=${SIZE}
NbChannels=3
BatchSize=${BATCH_SIZE}

[sp.Transformation-1]
Type=RescaleTransformation
Width=$(${SIZE} + ${MARGIN})
Height=$(${SIZE} + ${MARGIN})
KeepAspectRatio=1
ResizeToFit=0

[sp.Transformation-2]
Type=PadCropTransformation
Width=$(${SIZE} + ${MARGIN})
Height=$(${SIZE} + ${MARGIN})

[sp.Transformation-3]
Type=ColorSpaceTransformation
ColorSpace=BGR

[sp.Transformation-4]
Type=RangeAffineTransformation
FirstOperator=Minus
FirstValue=104.0 117.0 124.0 ; BGR format, same as in Caffe
SecondOperator=Divides
SecondValue=255.0

[sp.OnTheFlyTransformation-1]
Type=SliceExtractionTransformation
ApplyTo=LearnOnly
Width=[sp]SizeX
Height=[sp]SizeY
RandomOffsetX=1
RandomOffsetY=1

[sp.OnTheFlyTransformation-2]
Type=SliceExtractionTransformation
ApplyTo=TestOnly
Width=[sp]SizeX
Height=[sp]SizeY
OffsetX=$(${MARGIN} // 2)
OffsetY=$(${MARGIN} // 2)

[sp.OnTheFlyTransformation-3]
Type=FlipTransformation
ApplyTo=LearnOnly
RandomHorizontalFlip=1

[conv_def]
Type=Conv
ActivationFunction=Rectifier
WeightsFiller=HeFiller
ConfigSection=common.config

[conv1_1] conv_def
Input=sp
KernelDims=3 3
NbOutputs=64
Stride=1
Padding=1

[conv1_2] conv_def
Input=conv1_1
KernelDims=3 3
NbOutputs=64
Stride=1
Padding=1

[pool1]
Input=conv1_2
Type=Pool
PoolDims=2 2
NbOutputs=[conv1_2]NbOutputs
Stride=2
Pooling=Max
Mapping.Size=1

[conv2_1] conv_def
Input=pool1
KernelDims=3 3
NbOutputs=128
Stride=1
Padding=1

[conv2_2] conv_def
Input=conv2_1
KernelDims=3 3
NbOutputs=128
Stride=1
Padding=1

[pool2]
Input=conv2_2
Type=Pool
PoolDims=2 2
NbOutputs=[conv2_2]NbOutputs
Stride=2
Pooling=Max
Mapping.Size=1

[conv3_1] conv_def
Input=pool2
KernelDims=3 3
NbOutputs=256
Stride=1
Padding=1

[conv3_2] conv_def
Input=conv3_1
KernelDims=3 3
NbOutputs=256
Stride=1
Padding=1

[conv3_3] conv_def
Input=conv3_2
KernelDims=3 3
NbOutputs=256
Stride=1
Padding=1

[pool3]
Input=conv3_3
Type=Pool
PoolDims=2 2
NbOutputs=[conv3_3]NbOutputs
Stride=2
Pooling=Max
Mapping.Size=1

[conv4_1] conv_def
Input=pool3
KernelDims=3 3
NbOutputs=512
Stride=1
Padding=1

[conv4_2] conv_def
Input=conv4_1
KernelDims=3 3
NbOutputs=512
Stride=1
Padding=1

[conv4_3] conv_def
Input=conv4_2
KernelDims=3 3
NbOutputs=512
Stride=1
Padding=1

[pool4]
Input=conv4_3
Type=Pool
PoolDims=2 2
NbOutputs=[conv4_3]NbOutputs
Stride=2
Pooling=Max
Mapping.Size=1

[conv5_1] conv_def
Input=pool4
KernelDims=3 3
NbOutputs=512
Stride=1
Padding=1

[conv5_2] conv_def
Input=conv5_1
KernelDims=3 3
NbOutputs=512
Stride=1
Padding=1

[conv5_3] conv_def
Input=conv5_2
KernelDims=3 3
NbOutputs=512
Stride=1
Padding=1

[pool5]
Input=conv5_3
Type=Pool
PoolDims=2 2
NbOutputs=[conv5_3]NbOutputs
Stride=2
Pooling=Max
Mapping.Size=1

[fc6]
Input=pool5
Type=Fc
NbOutputs=4096
ActivationFunction=Rectifier
WeightsFiller=HeFiller
ConfigSection=common.config

[fc6.drop]
Input=fc6
Type=Dropout
NbOutputs=[fc6]NbOutputs
ConfigSection=fc6.dropconfig

[fc6.dropconfig]
Dropout=0.5

[fc7]
Input=fc6.drop
Type=Fc
NbOutputs=4096
ActivationFunction=Rectifier
WeightsFiller=HeFiller
BiasFiller=ConstantFiller

[fc7.drop]
Input=fc7
Type=Dropout
NbOutputs=[fc7]NbOutputs
ConfigSection=fc7.dropconfig

[fc7.dropconfig]
Dropout=0.5

[fc8]
Input=fc7.drop
Type=Fc
NbOutputs=1000
ActivationFunction=Linear
WeightsFiller=XavierFiller
ConfigSection=common.config

[soft1]
Input=fc8
Type=Softmax
NbOutputs=[fc8]NbOutputs
WithLoss=1

[soft1.Target]
TopN=5

[common.config]
NoBias=1
Solvers.LearningRate=${LR}
Solvers.Decay=${WD}
Solvers.Momentum=${MOMENTUM}
Solvers.LearningRatePolicy=StepDecay
Solvers.LearningRateStepSize=$([sp]_EpochSize * ${STEP_DECAY_EPOCHS})
Solvers.LearningRateDecay=${STEPDECAY_RATE}
;Solvers.IterationSize=16

[vtemplier]

Hi @prachikashikar,

I can observe in your ini file that the final FC layer [fc8] still have 1000 outputs while you are using the CIFAR10 dataset which has 10 classes. Thus, you should change NbOutputs from 1000 to 10.

Then, perhaps you will need to adapt some hyperparameters and some transformations to the new dataset. You can find an example at https://github.com/CEA-LIST/N2D2/blob/master/models/cifar-10.ini

Don't hesitate to tell us if all is ok

[vtemplier]

I suppose it is ok now and I will close this issue.

Don't hesitate to re-open it to provide more details.