Help with inference on Arduino (outputs are 0)

[expeon07]

Hi! I have a sequential model from Keras and I'm trying to deploy it to Arduino Uno using AIfES.

This is my model in Keras:

Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input (Dense)               (None, 10)                510       

 dense_2 (Dense)             (None, 10)                110       

 output (Dense)              (None, 5)                 55        

=================================================================
Total params: 675
Trainable params: 675
Non-trainable params: 0
________________________________________________________________

However, my outputs are 0. Can you check if my configuration in Arduino is correct?

// Tensor for the input data
  uint16_t input_shape[] = {DATA_COUNT, INPUTS};                      // Definition of the input shape
  aitensor_t input_tensor = AITENSOR_2D_F32(input_shape, input_data); // Creation of the input AIfES tensor with two dimensions and data type F32 (float32)

  // Tensor for the output data
  float output_data[DATA_COUNT * OUTPUTS];          // Output data
  uint16_t output_shape[2] = {DATA_COUNT, OUTPUTS}; // Definition of the output shape
  aitensor_t output_tensor = AITENSOR_2D_F32(output_shape, output_data);

  // ---------------------------------- Layer definition ---------------------------------------

  uint16_t input_layer_shape[] = {1, INPUTS};                                  // Definition of the input layer shape (The 1 must remain here regardless of the number of data sets)
  ailayer_input_f32_t input_layer = AILAYER_INPUT_F32_M(2, input_layer_shape); // Creation of the AIfES input layer
  ailayer_dense_f32_t dense_layer_1 = AILAYER_DENSE_F32_M(NEURONS_1, W0_data, b0_data);
  ailayer_relu_f32_t relu_layer_1 = AILAYER_RELU_F32_M(); // Hidden activation function
  ailayer_dense_f32_t dense_layer_2 = AILAYER_DENSE_F32_M(NEURONS_2, W1_data, b1_data);
  ailayer_relu_f32_t relu_layer_2 = AILAYER_RELU_F32_M(); // Output activation function
  ailayer_dense_f32_t output_layer = AILAYER_DENSE_F32_M(OUTPUTS, W2_data, b2_data);
  ailayer_relu_f32_t relu_output_layer = AILAYER_RELU_F32_M();

  ailoss_mse_t mse_loss; // Loss: mean squared error

  // --------------------------- Define the structure of the model ----------------------------

  aimodel_t model; // AIfES model
  ailayer_t *x;    // Layer object from AIfES, contains the layers

  // Passing the layers to the AIfES model
  model.input_layer = ailayer_input_f32_default(&input_layer);
  x = ailayer_dense_f32_default(&dense_layer_1, model.input_layer);
  x = ailayer_relu_f32_default(&relu_layer_1, x);
  x = ailayer_dense_f32_default(&dense_layer_2, x);
  x = ailayer_relu_f32_default(&relu_layer_2, x);
  x = ailayer_dense_f32_default(&output_layer, x);
  model.output_layer = ailayer_relu_f32_default(&relu_output_layer, x);

  // Add the loss to the AIfES model
  model.loss = ailoss_mse_f32_default(&mse_loss, model.output_layer);

  aialgo_compile_model(&model); // Compile the AIfES model

Thank you!!!

[PierreGembaczka]

Hi @expeon07 ,

If I understand the Keras summary correctly you have the following network: 50-10(relu)-10(relu)-5(relu)

This will be too large for the Uno. I built you a quick example for the Mega (Wokwi simulation): https://wokwi.com/projects/329367430475285076

I use the FlatWeight method here. So all weights in a row in one array. With so many weights it is much safer. Here is the link to the tutorial: https://create.arduino.cc/projecthub/aifes_team/aifes-inference-tutorial-f44d96

You have to put your weights in the "weights.h". I have only stored an array with ones here.

We will soon provide Python wrappers to export a model with one code line. Here is a code snippet how to extract the weights directly in the right order from your Keras model.

First the weight sort function:

import collections

# Convert the weights to AIfES FlatWeights
def flatten_weights(weights):
    w = []
    for l in weights:
        if isinstance(l, collections.Iterable):
            w = w + flatten_weights(l)
        else:
            w = w + [l]
    return w

And this is how you use it in Keras:

# Get weights
weights = model.get_weights()
aifes_flat_weights = flatten_weights(weights)

print("\n\n")
print("const float FlatWeights[] = {")

n = m = 0
while m < len(aifes_flat_weights):
    m = m + 5
    weight_line = 'f, '.join(map(str, aifes_flat_weights[n:m]))
    if m < len(aifes_flat_weights):
        weight_line = weight_line + "f,"
    else:
        weight_line = weight_line + "};\n\n"
    print(weight_line)
    n = m

Let us know if it worked 😀

[expeon07]

Hi, thanks a lot! I reduced the model and it fits the Uno now. I also spent some time trying to figure out how to use PROGMEM with my weights and biases into the model but I finally got it working by separating the arrays in another header file, as you mentioned.

I think a wrapper in python to export the model would be great!

[PierreGembaczka]

Hi @expeon07

great news. Awesome that it worked out. 👍

I don't know what your network is doing, but if you do a Hackster or Arduino Project, let me know.😀

[expeon07]

Hi @AIfES-Pierre, I'm using your framework for my thesis. I'll let you know when it's finished :D