multi-task learning using HLS4ML

[rtspk]

Hi Guys,

I am pretty excited to see this framework. Thank alot guys for putting a lot of effort into this framework development. The Jupyter Hub notebook tutorials are excellent.

I tried to convert a custom Lenet-like CNN model on mnist. It worked fine ( additional point: I did on locally on a laptop, with an older hls4ml version. I will switch to the new version now).

For my research, I have to make a CNN model for multi-task learning. ( Fig below)

1- Does the hls4ml framework currently support the conversion of a multi-task-learning Keras Model to the HLS4ML HLS Model?

• If it is not supported, then I am also willing to do explore what Customizations can be done to the hls4ml framework to make it work. Any guidance or direction on how to proceed with the possible customization would be really nice. 2- What is the biggest model that you guys ( or anyone else you know) have converted using hls4ml?
• Alexnet ( or similar ) , VGG16 ( or similar ), Mobile-net ( or similar) etc.
• This big model will serve as a backbone ( shared layers) in the multi-task-learning Keras model. 3- Does the framework currently supports Skip connections?

Thanks again guys

Best Regards Rizwan

[MODISENEISO]

Hello did you manage I am trying to convert a much complex model and running into loop unrolling excessive error

[jmitrevs]

For larger models with CNNs you should use io_type="io_stream". That also supports skip stops. For CNNs, io_type="io_parallel" is only for small models without skip connections, or MLPs.

[MODISENEISO]

For larger models with CNNs you should use io_type="io_stream". That also supports skip stops. For CNNs, io_type="io_parallel" is only for small models without skip connections, or MLPs.

I am using io_stream however I am getting this error in nnet_image_stream.h:36 Is this template necessary if images are already preprocessed

ERROR: [XFORM 203-504] Stop unrolling loop 'ResizeHeight' (firmware/nnet_utils/nnet_image_stream.h:36) in function 'nnet::resize_nearest<nnet::array<ap_fixed<16, 6, (ap_q_mode)5, (ap_o_mode)3, 0>, 64u>, config38>' because it may cause large runtime and excessive memory usage due to increase in code size. Please avoid unrolling the loop or form sub-functions for code in the loop body. ERROR: [HLS 200-70] Pre-synthesis failed. command 'ap_source' returned error code while executing "source [lindex $::argv 1] " ("uplevel" body line 1) invoked from within "uplevel #0 { source [lindex $::argv 1] } "

[MODISENEISO]

For larger models with CNNs you should use io_type="io_stream". That also supports skip stops. For CNNs, io_type="io_parallel" is only for small models without skip connections, or MLPs.

I am using io_stream however I am getting this error in nnet_image_stream.h:36 Is this template necessary if images are already preprocessed

ERROR: [XFORM 203-504] Stop unrolling loop 'ResizeHeight' (firmware/nnet_utils/nnet_image_stream.h:36) in function 'nnet::resize_nearest<nnet::array<ap_fixed<16, 6, (ap_q_mode)5, (ap_o_mode)3, 0>, 64u>, config38>' because it may cause large runtime and excessive memory usage due to increase in code size. Please avoid unrolling the loop or form sub-functions for code in the loop body. ERROR: [HLS 200-70] Pre-synthesis failed. command 'ap_source' returned error code while executing "source [lindex $::argv 1] " ("uplevel" body line 1) invoked from within "uplevel #0 { source [lindex $::argv 1] } "

I dont get the purpose of this resizing template if I have model trained for images with size 128*128 do I need this in model conversion

[MODISENEISO]

This is the config I am using