fastmachinelearning / hls4ml

Machine learning on FPGAs using HLS
https://fastmachinelearning.org/hls4ml
Apache License 2.0
1.24k stars 402 forks source link

Configuration editor #784

Open vloncar opened 1 year ago

vloncar commented 1 year ago

Description

Changing configuration in hls4ml requires deep understanding of the conversion process in order to be successful. New layers can be introduced, which pick up default precision definition causing problems for subsequent layers, unexpected allocation of DSPs etc. To solve this, users have to anticipate new layer names and with a meticulous trial-and-error procedure nail the initial configuration that will result in perfect conversion. This is way too cumbersome but until we have a better configuration system it is the only way. This PR proposes an alternative, to be able update the configuration of an existing hls4ml model interactively, via a GUI. It came out of a crazy idea I had last night after a few beers. See it in action:

Screenshot from 2023-05-08 21-26-23

It is very simple to use, make your changes and click "Update", the model will be updated and the changes should be reflected on the architecture plot on the left side.

This builds upon the recent automatic configuration support, by exposing only the configurable attributes (precision, strategy, RF etc). Some parts of it are a bit hacky for now, until we improve the configuration itself (ideally, we should have an model.get_config()/model.set_config() API).

The GUI is built using PySimpleGUI library, you can install it with pip install pysimplegui. Since the feature is optional, I also added an installation extra, so installation of pip install hls4ml[gui] will ensure the dependency is there. I considered a pure Tk implementation using tkinter from the standard library to avoid the new dependency, but the equivalent GUI had 400 lines of code and no support for mouse scroll events so I abandoned it.

The GUI is part of the utilities, to use it, run:

model = hls4ml.converters.convert_from_keras_model(...)
hls4ml.utils.edit_model_configuration(model)

A GUI should open up. If you use this through SSH or VS code server it may be slow to open. I don't know what is causing this, possibly an issue with the GUI library itself. Pure tkinter implementation didn't have this issue.

In the future, we can expand it to support creating the initial configuration, based on Keras/PyTorch/ONNX model.

Type of change

Also fixes a minor bug in definition of the trace attribute (was string, should be bool) and adds strategy attribute to supported layers.

Tests

We currently don't have any way of testing GUIs and this is a can of worms, so I suggest we skip it for now.

Checklist

jmitrevs commented 1 year ago

Can this work inside a jupyter notebook?