search

CEA-LIST / N2D2

N2D2 is an open source CAD framework for Deep Neural Network simulation and full DNN-based applications building.
Other
146 stars 35 forks source link

Export issue for C-int #57

Closed e-dupuis closed 3 years ago

e-dupuis commented 4 years ago

When exporting a trained net in C with the following command:

n2d2 cifar-10.ini -export C -nbbits 8 -calib -1

I have the following issue:

Option -export: generate an export and exit [C]
Option -nbbits: number of bits per weight for exports [8]
Option -calib: number of stimuli used for the calibration (0 = no calibration, -1 = use the full test dataset) [-1]
Loading network configuration file cifar-10.ini
sp.StimuliData-normalize processing 45000 stimuli
---
sp.StimuliData-normalize data:
Number of stimuli: 45000
Data width range: [32, 32]
Data height range: [32, 32]
Data channels range: [3, 3]
Value range: [0, 255]
Value mean: 120.736
Value std. dev.: 64.1757
---
Layer: conv1.1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv1.1.cfg
  # Shared synapses: 896
  # Virtual synapses: 848288
  # Inputs dims: 32 32 3 
  # Outputs dims: 32 32 32 
Layer: bn1.1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn1.1.cfg
  # Inputs dims: 32 32 32 
  # Outputs dims: 32 32 32 
Layer: conv1.2 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv1.2.cfg
  # Shared synapses: 9248
  # Virtual synapses: 9048096
  # Inputs dims: 32 32 32 
  # Outputs dims: 32 32 32 
Layer: bn1.2 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn1.2.cfg
  # Inputs dims: 32 32 32 
  # Outputs dims: 32 32 32 
Layer: pool1 [Pool(Frame_CUDA)]
Notice: Could not open configuration file: pool1.cfg
  # Inputs dims: 32 32 32 
  # Outputs dims: 16 16 32 
Layer: drop1 [Dropout(Frame_CUDA)]
Notice: Could not open configuration file: drop1.cfg
  # Inputs dims: 16 16 32 
  # Outputs dims: 16 16 32 
Layer: conv2.1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv2.1.cfg
  # Shared synapses: 18496
  # Virtual synapses: 4333632
  # Inputs dims: 16 16 32 
  # Outputs dims: 16 16 64 
Layer: bn2.1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn2.1.cfg
  # Inputs dims: 16 16 64 
  # Outputs dims: 16 16 64 
Layer: conv2.2 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv2.2.cfg
  # Shared synapses: 36928
  # Virtual synapses: 8667200
  # Inputs dims: 16 16 64 
  # Outputs dims: 16 16 64 
Layer: bn2.2 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn2.2.cfg
  # Inputs dims: 16 16 64 
  # Outputs dims: 16 16 64 
Layer: pool2 [Pool(Frame_CUDA)]
Notice: Could not open configuration file: pool2.cfg
  # Inputs dims: 16 16 64 
  # Outputs dims: 8 8 64 
Layer: drop2 [Dropout(Frame_CUDA)]
Notice: Could not open configuration file: drop2.cfg
  # Inputs dims: 8 8 64 
  # Outputs dims: 8 8 64 
Layer: conv3.1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv3.1.cfg
  # Shared synapses: 73856
  # Virtual synapses: 3965056
  # Inputs dims: 8 8 64 
  # Outputs dims: 8 8 128 
Layer: bn3.1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn3.1.cfg
  # Inputs dims: 8 8 128 
  # Outputs dims: 8 8 128 
Layer: conv3.2 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv3.2.cfg
  # Shared synapses: 147584
  # Virtual synapses: 7929984
  # Inputs dims: 8 8 128 
  # Outputs dims: 8 8 128 
Layer: bn3.2 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn3.2.cfg
  # Inputs dims: 8 8 128 
  # Outputs dims: 8 8 128 
Layer: pool3 [Pool(Frame_CUDA)]
Notice: Could not open configuration file: pool3.cfg
  # Inputs dims: 8 8 128 
  # Outputs dims: 4 4 128 
Layer: drop3 [Dropout(Frame_CUDA)]
Notice: Could not open configuration file: drop3.cfg
  # Inputs dims: 4 4 128 
  # Outputs dims: 4 4 128 
Layer: fc1 [Fc(Frame_CUDA)]
Notice: Could not open configuration file: fc1.cfg
  # Synapses: 20490
  # Inputs dims: 4 4 128 
  # Outputs dims: 1 1 10 
Layer: softmax [Softmax(Frame_CUDA)]
Notice: Could not open configuration file: softmax.cfg
  # Inputs dims: 1 1 10 
  # Outputs dims: 1 1 10 
Target: softmax (target value: 1 / default value: 0 / top-n value: 1)
Total number of neurons: 129044
Total number of nodes: 258068
Total number of synapses: 307498
Total number of virtual synapses: 34812746
Total number of connections: 34870090
Notice: Unused section conv_def in INI file
Notice: Unused section softmax.Target in INI file
CUDNN initialized
Learning database size: 45000 images
Validation database size: 5000 images
Testing database size: 10000 images
Importing weights from directory 'weights_validation'.
Remove Dropout...
  remove Dropout "drop1"
  remove Dropout "drop2"
  remove Dropout "drop3"
export_C_int8/stimuli_stats processing 5000 stimuli
Integer stimuli export with range > 1 (2.09213). Data will be truncated, possible data loss.
export_C_int8/stimuli_before_calibration_stats processing 5000 stimuli
Stimuli range before transformation is 2.09213
Load previously saved RangeStats and Histogram for the calibration? (y/n) n
Calculating calibration data range and histogram...
Notice: stimuli depth is 8U (according to database first stimulus)
CUBLAS initialized
Calibration data 128/5000
Calibration data 256/5000
Calibration data 320/5000
Calibration data 448/5000
Calibration data 512/5000
Calibration data 640/5000
Calibration data 704/5000
Calibration data 832/5000
Calibration data 960/5000
Calibration data 1024/5000
Calibration data 1152/5000
Calibration data 1216/5000
Calibration data 1344/5000
Calibration data 1408/5000
Calibration data 1536/5000
Calibration data 1600/5000
Calibration data 1728/5000
Calibration data 1856/5000
Calibration data 1920/5000
Calibration data 2048/5000
Calibration data 2112/5000
Calibration data 2240/5000
Calibration data 2304/5000
Calibration data 2432/5000
Calibration data 2560/5000
Calibration data 2624/5000
Calibration data 2752/5000
Calibration data 2816/5000
Calibration data 2944/5000
Calibration data 3008/5000
Calibration data 3136/5000
Calibration data 3200/5000
Calibration data 3328/5000
Calibration data 3456/5000
Calibration data 3520/5000
Calibration data 3648/5000
Calibration data 3712/5000
Calibration data 3840/5000
Calibration data 3904/5000
Calibration data 4032/5000
Calibration data 4160/5000
Calibration data 4224/5000
Calibration data 4352/5000
Calibration data 4416/5000
Calibration data 4544/5000
Calibration data 4608/5000
Calibration data 4736/5000
Calibration data 4800/5000
Calibration data 4928/5000
Calibration data 4992/5000
Calibration (8 bits):
conv1.1: scalingFactor = 6.952   
bn1.1: scalingFactor = 1.697   
conv1.2: scalingFactor = 48.64   
bn1.2: scalingFactor = 1.961   
conv2.1: scalingFactor = 5.154   
bn2.1: scalingFactor = 1.973   
conv2.2: scalingFactor = 16.68   
bn2.2: scalingFactor = 5.125   
conv3.1: scalingFactor = 21.24   
bn3.1: scalingFactor = 12.22   
conv3.2: scalingFactor = 43.8   
bn3.2: scalingFactor = 3.016   
fc1: scalingFactor = 26.62   
Skipping normalization of cell softmax.
export_C_int8/stimuli_stats processing 10000 stimuli
Integer stimuli export with range > 1 (2.092). Data will be truncated, possible data loss.
Exporting Test dataset to "export_C_int8/stimuli"....................
Generating C export to "export_C_int8":
-> Generating cell conv1.1
-> Generating cell bn1.1
Time elapsed: 56.75 s
Error: Can't export a non-integral floating-point as an integral parameter. The network must be quantized beforehand with the -calib option if an integer export is necessary.

I don't understand because the argument -calib -1 is present, and even with different value than -1 I have the same issue, But I encountered this issue only for this network, for LeNet it is working fine

Djip007 commented 4 years ago

Not 100% sure but not all Cell type can be quantized... look to be the case for BatchNorm. but the authors can give more advise.

davidbriand-cea commented 4 years ago

Hi Etienne, This error occur because it is not possible to quantize batchnormalization layer. If you want to quantize a model with batchnormalization layers you have to use the command "-fuse". The batch normalization parameters are then automatically fused with convolutions parameters with this command.

Moreover you have to take care to add the non-linear activations functions (e.g Relu..) on the batchnormalizations layers instead on the convolutions layers.

olivierbichler-cea commented 3 years ago

Closing this issue, the BatchNorm fuse with Conv is now automatic