search

fastmachinelearning / hls4ml

Machine learning on FPGAs using HLS
https://fastmachinelearning.org/hls4ml
Apache License 2.0
1.27k stars 409 forks source link

Dense layer array partition issue #304

Closed waldenou closed 3 years ago

waldenou commented 3 years ago

Hi,

I'm trying to migrate the generated firmware for my keras model to Alveo U250 card using Vitis. I have attached my model definition below. I was able to do that with Strategy: Resource but not Strategy: Latency due to the limit of array partition size in Vitis, which is 1024 instead of 4096 (in Vivado). The synthesis would fail. I was wondering if there is a way to resolve this issue besides reducing the dense layer size. Is it possible to use Strategy: Resource for the dense layer only? And the rest of the model can use Strategy: Latency. Any help is appreciated!

model.zip

vloncar commented 3 years ago

Hi,

Thanks for trying out hls4ml. Unfortunately, Vitis HLS is not supported yet (you'll run into more issues beyond that one). Please use Vivado HLS (available in 2020.1).

Regarding Resource strategy, you can specify per-layer strategies. Check out the docs on this topic.

Cheers, V.

brtsfr commented 3 months ago

hi @vloncar, how can i set per layer parameters like precision and strategy, when the pytorch model is used? i only found tutorials about keras model. After i tried "config['LayerName']['conv0_0']['Precision'] = 'ap_fixed<8,6>'" like #903, hls4ml just returned KeyError: LayerName.

JanFSchulte commented 3 months ago

There should be no difference in the way these settings work between using hls4ml for keras and pytorch models. To me it looks like you using the string 'Layername' to access the config for a given layer, while it should be string equal to the name of the layer.

brtsfr commented 3 months ago

yes, i had the same thought as you. so i removed ['LayerName'], it becomes config['conv0_0']['Precision'] = 'ap_fixed<8,6>'. But i still got KeyError: 'conv0_0'. Here the layer name 'conv0_0' i was got from this command: hls4ml.utils.plot_model(hls_model, show_shapes=True, show_layer_names=True), i attached the image. So i think the name is correct. model_structure_name

JanFSchulte commented 3 months ago

Easiest thing to debug would probably be to just print out the config and see how it actually looks like.

brtsfr commented 3 months ago

ok. i tried: plotting.print_dict(config). But returned AttributeError: module plotting has no attribute print_dict... How can i print out the config?

brtsfr commented 3 months ago

oh, i fotgot to mention that i am using a loaded pytorch model.

JanFSchulte commented 3 months ago

I was just thinking about a simple print (config), the config is a simple python dictionary. But I just tried it locally and realized that i was wrong, granularities of the config below model are currently not implemented in the pytorch parser, you can only configure individual layers for keras models. Goes on the to-do list, I'll try to implement that quickly.

brtsfr commented 3 months ago

yes, i also tried print(config), but no related info. i appreciate your contribution to that. i will try to use keras model for now.

brtsfr commented 2 months ago

hi @JanFSchulte, about granularities i wrote config['Model']['Strategy'] = 'Resource', avoid to set it below model, is it correct? And may i ask that how to set Trace for profiling in pytorch parser? I've tried config['Model']['Trace'] = True or for layer in config['LayerName'].keys(): config['LayerName'][layer]['Trace'] = True, like it is with keras, but neither of them is right...