Problems during calibration of 1d sound network

[andreistoian]

Hi,

I have a 1D sound network that takes 8000x1 size inputs. I'm trying to export to int8 with calibration. I'm using the speech_commands dataset (following the example - using DIR_Database). However, the data in the dataset is at 16khz thus giving 16000x1.

I used the following ini file:

; Database
[database]
Type=DIR_Database
DataPath=...Documents/sound1d/speech_commands
ValidExtensions=wav
IgnoreMasks=*/_background_noise_
Learn=0.6
Validation=0.2
Depth=3

; Environment
[sp]
Size=8000 1 1
BatchSize=${BATCH_SIZE}

[sp.TransformationRescale]
Type=RescaleTransformation
Width=8000
Height=1
KeepAspectRatio=0
ResizeToFit=0

; Here, we insert an ONNX graph in the N2D2 flow the same way as a regular Cell
[onnx]
Input=sp
Type=ONNX
File=.../Documents/sound1d/sound_pytorch.onnx

[37.Target]
TopN=5

• The StimuliProvider input size is un-intuitive. The data is 16000x1 but I need to set Size=8000 1 1 in the [SP] section since the transformation (Rescale) does not alter the StimuliProvider size. If I set Size=16000 1 1 in the [sp] section, the ONNX parser throws an error saying that the stimuli provider size and the onnx input size do not match. The solution I currently use is a rather non-intuitive work around
• The calibration crashes in the convCell_Frame::forward .

  void N2D2::ConvCell_Frame_Kernels::forward(const T* alpha,
                                         const Tensor<T>& inputs,
                                         const Tensor
                                         <T>& sharedSynapses,
                                         const Descriptor& desc,
                                         const T* beta,
                                         Tensor<T>& outputs,
                                         const Tensor<bool>& maps)
  {
  const unsigned int oxSize
      = (unsigned int)((inputs.dimX() + desc.padding[0] + desc.padding[2]
                        - sharedSynapses.dimX() + desc.stride[0])
                       / (double)desc.stride[0]);

since desc.padding has only size 2 so this line crashes (desc.padding[2] is OOB). It seems the mConvDesc in ConvCell_Frame is not handled appropriately for 1D convolution, only the base ConvCell class does the appropriate massaging of the dimensions (adding a 1 to the 2d dimension when the convolution is 1d)

      mConvDesc(subSampleDims, strideDims, paddingDims, dilationDims)
{

[olivierbichler-cea]

Regarding the crash, that was indeed an oversight from the previous patch I committed for 1D support. This should be fixed now in the latest commit.

Regarding the StimuliProvider, the confusion comes from the fact that the size you specify is its output size, not its input size. Like any other layer, you always specify the output size. The input size is always unspecified. For the StimuliProvider, it can be anything coming from a Database, which does not enforce any size. It is therefore your responsibility to ensure that the data size matches the specified output size of the StimuliProvider, after passing through all the transformations attached to it.

[andreistoian]

Thanks.

Calibration and export to int8 now works with 1D convolution. I still need to test with real labels to compare accuracy but the export compiles and runs well.