search

CEA-LIST / N2D2

N2D2 is an open source CAD framework for Deep Neural Network simulation and full DNN-based applications building.
Other
146 stars 36 forks source link

Floating point exception when exporting MobileNet in C #85

Closed FabriceAuz closed 3 years ago

FabriceAuz commented 3 years ago

I tried to export a trained MobileNet network using the commant line ./n2d2 "./MobileNet_v1_batchnorm_export_DNeuro.ini" -seed 1 -test -w /dev/null -export DNeuro_V2 -fuse -nbbits 8 -calib -1 -act-clipping-mode KL-Divergence -db-export 10 -export-parameters DNeuro1.ini but I get a floating point exception. Here is the complete output from n2d2:

Option -seed: N2D2 random seed (0 = time based) [1]
Option -test: perform testing
Option -fuse: fuse BatchNorm with Conv for test and export
Option -export: generate an export and exit [DNeuro_V2]
Option -nbbits: number of bits per weight for exports [8]
Option -calib: number of stimuli used for the calibration (0 = no calibration, -1 = use the full test dataset) [-1]
Option -act-clipping-mode: activation clipping mode on export, can be 'None', 'MSE' or 'KL-Divergence' [KL-Divergence]
Option -w: start with weights imported from a specified location (even when loading a previously saved state) [/dev/null]
Option -db-export: max. number of stimuli to export (0 = no dataset export, -1 = unlimited) [10]
Option -export-parameters: parameters for export [DNeuro1.ini]
Loading network configuration file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv1_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv1_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv2_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv2_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv3_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv3_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv4_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv4_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv5_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv5_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv6_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv6_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv7_1_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv7_1_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv7_2_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv7_2_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv7_3_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv7_3_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv7_4_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv7_4_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv7_5_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv7_5_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv8_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv8_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv9_3x3_dw] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Warning: overriding property ActivationFunction (previous value was "Rectifier") in section [conv9_1x1] in INI file ./MobileNet_v1_batchnorm_export_DNeuro.ini
Loading directory database "/local/DATABASE/isic_classification/training"
Loading directory database "/local/DATABASE/isic_classification/training/0"
Found 3322 stimuli
Loading directory database "/local/DATABASE/isic_classification/training/1"
Found 13497 stimuli
Loading directory database "/local/DATABASE/isic_classification/training/2"
Found 15920 stimuli
Loading directory database "/local/DATABASE/isic_classification/training/3"
Found 16547 stimuli
Loading directory database "/local/DATABASE/isic_classification/training/4"
Found 18571 stimuli
Loading directory database "/local/DATABASE/isic_classification/training/5"
Found 18720 stimuli
Loading directory database "/local/DATABASE/isic_classification/training/6"
Found 18883 stimuli
Loading directory database "/local/DATABASE/isic_classification/training/7"
Found 19331 stimuli
Found 19331 stimuli
Loading directory database "/local/DATABASE/isic_classification/validation"
Loading directory database "/local/DATABASE/isic_classification/validation/0"
Found 19531 stimuli
Loading directory database "/local/DATABASE/isic_classification/validation/1"
Found 19981 stimuli
Loading directory database "/local/DATABASE/isic_classification/validation/2"
Found 20131 stimuli
Loading directory database "/local/DATABASE/isic_classification/validation/3"
Found 20171 stimuli
Loading directory database "/local/DATABASE/isic_classification/validation/4"
Found 20271 stimuli
Loading directory database "/local/DATABASE/isic_classification/validation/5"
Found 20286 stimuli
Loading directory database "/local/DATABASE/isic_classification/validation/6"
Found 20301 stimuli
Loading directory database "/local/DATABASE/isic_classification/validation/7"
Found 20331 stimuli
Found 20331 stimuli
Loading directory database "/local/DATABASE/isic_classification/test"
Loading directory database "/local/DATABASE/isic_classification/test/0"
Found 21331 stimuli
Loading directory database "/local/DATABASE/isic_classification/test/1"
Found 23581 stimuli
Loading directory database "/local/DATABASE/isic_classification/test/2"
Found 24331 stimuli
Loading directory database "/local/DATABASE/isic_classification/test/3"
Found 24531 stimuli
Loading directory database "/local/DATABASE/isic_classification/test/4"
Found 25031 stimuli
Loading directory database "/local/DATABASE/isic_classification/test/5"
Found 25106 stimuli
Loading directory database "/local/DATABASE/isic_classification/test/6"
Found 25181 stimuli
Loading directory database "/local/DATABASE/isic_classification/test/7"
Found 25331 stimuli
Found 25331 stimuli
sp.StimuliData-raw processing 19331 stimuli
---
sp.StimuliData-raw data:
Number of stimuli: 19331
Data width range: [576, 1024]
Data height range: [450, 1024]
Data channels range: [3, 3]
Value range: [0, 255]
Value mean: 140.381
Value std. dev.: 60.6431
---
Layer: conv1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv1.cfg
  # Shared synapses: 864
  # Virtual synapses: 10773600
  # Inputs dims: 224 224 3
  # Outputs dims: 112 112 32
Layer: bn1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn1.cfg
  # Inputs dims: 112 112 32
  # Outputs dims: 112 112 32
Layer: conv1_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv1_3x3_dw.cfg
  # Shared synapses: 288
  # Virtual synapses: 3569792
  # Inputs dims: 112 112 32
  # Outputs dims: 112 112 32
Layer: bn1_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn1_3x3_dw.cfg
  # Inputs dims: 112 112 32
  # Outputs dims: 112 112 32
Layer: conv1_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv1_1x1.cfg
  # Shared synapses: 2048
  # Virtual synapses: 25690112
  # Inputs dims: 112 112 32
  # Outputs dims: 112 112 64
Layer: bn1_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn1_1x1.cfg
  # Inputs dims: 112 112 64
  # Outputs dims: 112 112 64
Layer: conv2_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv2_3x3_dw.cfg
  # Shared synapses: 576
  # Virtual synapses: 1784896
  # Inputs dims: 112 112 64
  # Outputs dims: 56 56 64
Layer: bn2_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn2_3x3_dw.cfg
  # Inputs dims: 56 56 64
  # Outputs dims: 56 56 64
Layer: conv2_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv2_1x1.cfg
  # Shared synapses: 8192
  # Virtual synapses: 25690112
  # Inputs dims: 56 56 64
  # Outputs dims: 56 56 128
Layer: bn2_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn2_1x1.cfg
  # Inputs dims: 56 56 128
  # Outputs dims: 56 56 128
Layer: conv3_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv3_3x3_dw.cfg
  # Shared synapses: 1152
  # Virtual synapses: 3527168
  # Inputs dims: 56 56 128
  # Outputs dims: 56 56 128
Layer: bn3_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn3_3x3_dw.cfg
  # Inputs dims: 56 56 128
  # Outputs dims: 56 56 128
Layer: conv3_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv3_1x1.cfg
  # Shared synapses: 16384
  # Virtual synapses: 51380224
  # Inputs dims: 56 56 128
  # Outputs dims: 56 56 128
Layer: bn3_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn3_1x1.cfg
  # Inputs dims: 56 56 128
  # Outputs dims: 56 56 128
Layer: conv4_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv4_3x3_dw.cfg
  # Shared synapses: 1152
  # Virtual synapses: 881792
  # Inputs dims: 56 56 128
  # Outputs dims: 28 28 128
Layer: bn4_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn4_3x3_dw.cfg
  # Inputs dims: 28 28 128
  # Outputs dims: 28 28 128
Layer: conv4_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv4_1x1.cfg
  # Shared synapses: 32768
  # Virtual synapses: 25690112
  # Inputs dims: 28 28 128
  # Outputs dims: 28 28 256
Layer: bn4_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn4_1x1.cfg
  # Inputs dims: 28 28 256
  # Outputs dims: 28 28 256
Layer: conv5_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv5_3x3_dw.cfg
  # Shared synapses: 2304
  # Virtual synapses: 1721344
  # Inputs dims: 28 28 256
  # Outputs dims: 28 28 256
Layer: bn5_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn5_3x3_dw.cfg
  # Inputs dims: 28 28 256
  # Outputs dims: 28 28 256
Layer: conv5_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv5_1x1.cfg
  # Shared synapses: 65536
  # Virtual synapses: 51380224
  # Inputs dims: 28 28 256
  # Outputs dims: 28 28 256
Layer: bn5_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn5_1x1.cfg
  # Inputs dims: 28 28 256
  # Outputs dims: 28 28 256
Layer: conv6_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv6_3x3_dw.cfg
  # Shared synapses: 2304
  # Virtual synapses: 430336
  # Inputs dims: 28 28 256
  # Outputs dims: 14 14 256
Layer: bn6_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn6_3x3_dw.cfg
  # Inputs dims: 14 14 256
  # Outputs dims: 14 14 256
Layer: conv6_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv6_1x1.cfg
  # Shared synapses: 131072
  # Virtual synapses: 25690112
  # Inputs dims: 14 14 256
  # Outputs dims: 14 14 512
Layer: bn6_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn6_1x1.cfg
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: conv7_1_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv7_1_3x3_dw.cfg
  # Shared synapses: 4608
  # Virtual synapses: 819200
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: bn7_1_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn7_1_3x3_dw.cfg
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: conv7_1_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv7_1_1x1.cfg
  # Shared synapses: 262144
  # Virtual synapses: 51380224
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: bn7_1_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn7_1_1x1.cfg
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: conv7_2_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv7_2_3x3_dw.cfg
  # Shared synapses: 4608
  # Virtual synapses: 819200
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: bn7_2_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn7_2_3x3_dw.cfg
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: conv7_2_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv7_2_1x1.cfg
  # Shared synapses: 262144
  # Virtual synapses: 51380224
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: bn7_2_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn7_2_1x1.cfg
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: conv7_3_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv7_3_3x3_dw.cfg
  # Shared synapses: 4608
  # Virtual synapses: 819200
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: bn7_3_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn7_3_3x3_dw.cfg
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: conv7_3_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv7_3_1x1.cfg
  # Shared synapses: 262144
  # Virtual synapses: 51380224
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: bn7_3_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn7_3_1x1.cfg
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: conv7_4_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv7_4_3x3_dw.cfg
  # Shared synapses: 4608
  # Virtual synapses: 819200
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: bn7_4_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn7_4_3x3_dw.cfg
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: conv7_4_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv7_4_1x1.cfg
  # Shared synapses: 262144
  # Virtual synapses: 51380224
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: bn7_4_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn7_4_1x1.cfg
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: conv7_5_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv7_5_3x3_dw.cfg
  # Shared synapses: 4608
  # Virtual synapses: 819200
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: bn7_5_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn7_5_3x3_dw.cfg
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: conv7_5_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv7_5_1x1.cfg
  # Shared synapses: 262144
  # Virtual synapses: 51380224
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: bn7_5_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn7_5_1x1.cfg
  # Inputs dims: 14 14 512
  # Outputs dims: 14 14 512
Layer: conv8_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv8_3x3_dw.cfg
  # Shared synapses: 4608
  # Virtual synapses: 204800
  # Inputs dims: 14 14 512
  # Outputs dims: 7 7 512
Layer: bn8_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn8_3x3_dw.cfg
  # Inputs dims: 7 7 512
  # Outputs dims: 7 7 512
Layer: conv8_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv8_1x1.cfg
  # Shared synapses: 524288
  # Virtual synapses: 25690112
  # Inputs dims: 7 7 512
  # Outputs dims: 7 7 1024
Layer: bn8_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn8_1x1.cfg
  # Inputs dims: 7 7 1024
  # Outputs dims: 7 7 1024
Layer: conv9_3x3_dw [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv9_3x3_dw.cfg
  # Shared synapses: 9216
  # Virtual synapses: 369664
  # Inputs dims: 7 7 1024
  # Outputs dims: 7 7 1024
Layer: bn9_3x3_dw [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn9_3x3_dw.cfg
  # Inputs dims: 7 7 1024
  # Outputs dims: 7 7 1024
Layer: conv9_1x1 [Conv(Frame_CUDA)]
Notice: Could not open configuration file: conv9_1x1.cfg
  # Shared synapses: 1048576
  # Virtual synapses: 51380224
  # Inputs dims: 7 7 1024
  # Outputs dims: 7 7 1024
Layer: bn9_1x1 [BatchNorm(Frame_CUDA)]
Notice: Could not open configuration file: bn9_1x1.cfg
  # Inputs dims: 7 7 1024
  # Outputs dims: 7 7 1024
Layer: pool [Pool(Frame_CUDA)]
Notice: Could not open configuration file: pool.cfg
  # Inputs dims: 7 7 1024
  # Outputs dims: 1 1 1024
Layer: fc [Fc(Frame_CUDA)]
Notice: Could not open configuration file: fc.cfg
  # Synapses: 8200
  # Inputs dims: 1 1 1024
  # Outputs dims: 1 1 8
Target: fc (target value: 1 / default value: 0 / top-n value: 5)
Layer: softmax [Softmax(Frame_CUDA)]
Notice: Could not open configuration file: softmax.cfg
  # Inputs dims: 1 1 8
  # Outputs dims: 1 1 8
Total number of neurons: 5042704
Total number of nodes: 10086416
Total number of synapses: 3193288
Total number of virtual synapses: 566859944
Total number of connections: 566910120
Notice: Unused section conv_def in INI file
CUDNN initialized
Learning database size: 19331 images
Validation database size: 1000 images
Testing database size: 5000 images
Notice: stimuli depth is 8U (according to database first stimulus)
Remove Dropout...
Fuse BatchNorm with Conv...
  fuse BatchNorm "bn1" with Conv "conv1"
    zero-variance conv1[0]
    zero-variance conv1[1]
    zero-variance conv1[2]
    zero-variance conv1[3]
    zero-variance conv1[4]
    zero-variance conv1[5]
    zero-variance conv1[6]
    zero-variance conv1[7]
    zero-variance conv1[8]
    zero-variance conv1[9]
    zero-variance conv1[10]
    zero-variance conv1[11]
    zero-variance conv1[12]
    zero-variance conv1[13]
    zero-variance conv1[14]
    zero-variance conv1[15]
    zero-variance conv1[16]
    zero-variance conv1[17]
    zero-variance conv1[18]
    zero-variance conv1[19]
    zero-variance conv1[20]
    zero-variance conv1[21]
    zero-variance conv1[22]
    zero-variance conv1[23]
    zero-variance conv1[24]
    zero-variance conv1[25]
    zero-variance conv1[26]
    zero-variance conv1[27]
    zero-variance conv1[28]
    zero-variance conv1[29]
    zero-variance conv1[30]
    zero-variance conv1[31]
Floating point exception [floating point invalid operation]
backtrace() returned 7 addresses
./n2d2(_ZN4N2D216exceptionHandlerEiP9siginfo_tPv+0x35)[0x556362609405]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12890)[0x7f7399ec5890]
./n2d2(_ZN4N2D27DeepNet21fuseBatchNormWithConvEv+0xdca)[0x55636236c7aa]
./n2d2(_Z14generateExportRK7OptionsRSt10shared_ptrIN4N2D27DeepNetEE+0x70)[0x556361fd51b0]
./n2d2(main+0x5d8)[0x556361f8a998]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f7374a4bb97]
./n2d2(_start+0x2a)[0x556361fccf8a]
olivierbichler-cea commented 3 years ago

Hi, This should be fixed in the latest commit. The issue was that the variance of the BatchNorm is < 1e-12 for all output channels, thus a division by zero was occurring. Now the export should run correctly, however the fact that the variance is < 1e-12 for all channels indicates that there may be an issue with the parameters of the network. Was the network correctly trained? Closing for now, please feel free to re-open the issue if you feel that it is not properly solved.