fastmachinelearning / hls4ml

Machine learning on FPGAs using HLS
https://fastmachinelearning.org/hls4ml
Apache License 2.0
1.27k stars 409 forks source link

Why nnet_pooling_stream.h does not support padding='same'? #617

Open loubinglei opened 2 years ago

loubinglei commented 2 years ago

I have a very simple CNN in TensorFlow and the padding strategy is 'same'. When it is converted to the CNN with io_tyoe = 'io_stream', one of the pad_bottom and pad_right are equal to 1. Then there would be a problem caused by this assert operation in nnet_pooling_stream.h. -> assert(CONFIG_T::pad_top == 0 && CONFIG_T::pad_bottom == 0 && CONFIG_T::pad_left == 0 && CONFIG_T::pad_right == 0);

the ERROR is: "_python3: firmware/nnet_utils/nnet_pooling_stream.h:227: void nnet::pooling2d_buffer_cl(hls::stream&, hls::stream&) [with data_T = nnet::array<ap_fixed<40, 20, AP_RND, AP_SAT>, 16>; res_T = nnet::array<ap_fixed<40, 20>, 16>; CONFIG_T = config4]: Assertion `CONFIG_T::pad_top == 0 && CONFIG_T::pad_bottom == 0 && CONFIG_T::pad_left == 0 && CONFIG_T::padright == 0' failed. Aborted (core dumped) " So my question is does hls4ml(v0.6.0) not support the padding strategy is 'same' for io_tyoe = 'io_stream' case?

1. generated config information by hls4ml

// max_pooling2d_5
struct config7 : nnet::pooling2d_config {
    static const unsigned in_height = OUT_HEIGHT_5;
    static const unsigned in_width = OUT_WIDTH_5;
    static const unsigned n_filt = N_FILT_7;
    static const unsigned stride_height = 2;
    static const unsigned stride_width = 2;
    static const unsigned pool_height = 2;
    static const unsigned pool_width = 2;
    static const unsigned filt_height = 2;
    static const unsigned filt_width = 2;
    static const unsigned n_chan = N_FILT_7;
    static const unsigned out_height = OUT_HEIGHT_7;
    static const unsigned out_width = OUT_WIDTH_7;
    static const unsigned pad_top = 0;
    static const unsigned pad_bottom = 1;
    static const unsigned pad_left = 0;
    static const unsigned pad_right = 1;
    static const nnet::Pool_Op pool_op = nnet::Max;
    static const nnet::conv_implementation implementation = nnet::conv_implementation::linebuffer;
    static const unsigned reuse = 1;
    typedef ap_fixed<32,16> accum_t;
};

2. CNN Code

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D, Input, Softmax

def create_model(num_layers):
    model = Sequential()
    model.add(Input(shape=(9,9,1))) 
    model.add(Conv2D(16, (3,3), input_shape = (9,9,1), activation='relu', padding='same'))
    model.add(MaxPooling2D(pool_size=(2,2), padding='same'))
    for _ in range(num_layers-1):
        model.add(Conv2D(16, (3,3), activation='relu', padding='same'))
        model.add(MaxPooling2D(pool_size=(2,2), padding='same'))
    model.add(Flatten())
    model.add(Dense(16, activation='relu'))
    model.add(Dense(2))
    model.add(Softmax(name='output_softmax'))
    model.compile(optimizer = "adam",metrics=['accuracy'], loss='categorical_crossentropy')
    return model

model = create_model(num_layers=5)
adarsh9975 commented 1 year ago

Hi, Were you able to solve this issue ? I have the same issue with hls4ml 0.7.1