Open sei-jgwohlbier opened 1 week ago
Hi! Thanks for testing this and working on a fix! I think your development goes into a promising direction. From a quick check of your fork, it seems to be like hsl4ml is still treating the new Constant layer like an input layer and is trying to find it's tensor at runtime. I'm not quite sure how best to fix it, but maybe @vloncar can advise.
Prerequisites
Please make sure to check off these prerequisites before submitting a bug report.
Quick summary
Merge layers that have a constant input do not work with PyTorch.
Details
Steps to Reproduce
import numpy as np import os import shutil import torch import torch.nn as nn from torchinfo import summary
from hls4ml.converters import convert_from_pytorch_model from hls4ml.utils.config import config_from_pytorch_model
test_root_path = Path(file).parent
if name == "main":
========================================================================================== Layer (type:depth-idx) Output Shape Param #
test [2, 2, 8] 12 ├─AvgPool1d: 1-1 [2, 2, 4] --
Total params: 12 Trainable params: 12 Non-trainable params: 0 Total mult-adds (M): 0
Input size (MB): 0.00 Forward/backward pass size (MB): 0.00 Params size (MB): 0.00 Estimated Total Size (MB): 0.00
{'Model': {'Precision': 'ap_fixed<16,6>', 'ReuseFactor': 1, 'ChannelsLastConversion': 'internal', 'TransposeOutputs': False, 'Strategy': 'Resource'}, 'PytorchModel': test( (conv1): Conv1d(2, 2, kernel_size=(3,), stride=(1,), padding=(1,), bias=False) (downsample): AvgPool1d(kernel_size=(1,), stride=(2,), padding=(0,)) ), 'InputShape': (None, 2, 8)} /home/hls4ml-user/work/ewstapp_research/isolate/NETWORK/hls4mlprj_mul_Vitis_io_stream Interpreting Model ... Topology: Layer name: downsample, layer type: AveragePooling1D, input shape: [[None, 2, 8]] Layer name: mul, layer type: Merge, input shape: [[None, 2, 4]] Layer name: cat, layer type: Concatenate, input shape: [[None, 2, 4], [None, 2, 4]] Creating HLS model WARNING: Changing pipeline style to "dataflow". Traceback (most recent call last): File "/home/hls4ml-user/work/ewstapp_research/isolate/NETWORK/test_mul.py", line 81, in
hls_model = convert_from_pytorch_model(
File "/home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/hls4ml/converters/init.py", line 308, in convert_from_pytorch_model
return pytorch_to_hls(config)
File "/home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/hls4ml/converters/pytorch_to_hls.py", line 374, in pytorch_to_hls
hls_model = ModelGraph(config, layer_list, inputs=input_layers)
File "/home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/hls4ml/model/graph.py", line 387, in init
self._make_graph(layer_list)
File "/home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/hls4ml/model/graph.py", line 416, in _make_graph
self.graph[name] = self.make_node(kind, name, layer, inputs, outputs)
File "/home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/hls4ml/model/graph.py", line 503, in make_node
node = layer_cls(self, name, attributes, inputs, outputs)
File "/home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/hls4ml/model/layers.py", line 117, in init
self.initialize()
File "/home/hls4ml-user/miniconda3/envs/hls4ml/lib/python3.10/site-packages/hls4ml/model/layers.py", line 950, in initialize
assert len(self.inputs) == 2