fastmachinelearning / hls4ml

Machine learning on FPGAs using HLS
https://fastmachinelearning.org/hls4ml
Apache License 2.0
1.29k stars 419 forks source link

Add LayerNorm support for Vivado #1110

Open rianbrooksflynn opened 3 weeks ago

rianbrooksflynn commented 3 weeks ago

Description

This PR adds support for Layer Normalization using either Keras or PyTorch with the Vivado backend in io_parallel mode.

This implementation uses a lookup table for inverse square root; the inputs to the lookup table follow a logarithmic distribution for better accuracy.

Tests have been added for both Keras and Pytorch parsing.

Credit is due to @Ethan0Jiang and @LostEcho365 (Zhixing Jiang and Dennis Yin) for their Vivado implementation and Keras parsing support; my contributions were making a change to the inverse square root lookup table implementation, implementing PyTorch support, and adding unit tests. (Here's a link to their pre-print.) The original code authors have given permission for their code to be merged into hls4ml.

Linked issue: https://github.com/fastmachinelearning/hls4ml/issues/1109

Type of change

Tests

Two unit tests added: test/pytest/test_layernorm.py and test/pytest/test_layernorm_pytorch.py

Checklist

JanFSchulte commented 3 weeks ago

The pytest failure is the QKeras tests timing out. There's 299 tests being run in that batch, which I guess is too many. Is there a why to reshuffle the batches to avoid the timeout?