fastmachinelearning / hls4ml

Machine learning on FPGAs using HLS
https://fastmachinelearning.org/hls4ml
Apache License 2.0
1.17k stars 387 forks source link

Keras Reshape Layer is Built with Error #932

Open MrFaith2001 opened 7 months ago

MrFaith2001 commented 7 months ago

Quick summary

I want to synthesize a model with reshape and concatenate layers through hls4ml, but when I build the generated HLS file, there will be a problem of undefined variables, do I need to manually set some additional configuration?

Details

My model consists of two inputs and an output, both of which pass through the corresponding reshape layer, and finally merge through the concatenate layer.

Steps to Reproduce

  1. Run conversion on model file with code My model file is here https://drive.google.com/file/d/1q0JdEZhn2AYalodmhS665kSUdEVZccOC/view?usp=drive_link
  import os

  os.environ["PATH"] += os.pathsep + "/data2t1/dataset/Xilinx_Public/Vivado/2019.2/bin"

  from tensorflow.keras.models import load_model
  import hls4ml
  from print_dict import print_dict
  from tensorflow import keras

  import tensorflow as tf
  from tensorflow.keras.models import Model

  model_name = "Ret_re5re6_in_box_con4_out"
  model = load_model('./model/{}.h5'.format(model_name))
  model.summary()
  model.compile()

  import plotting
  import matplotlib.pyplot as plt
  from sklearn.metrics import accuracy_score
  import hls4ml
  from pathlib import Path

  hls_config = hls4ml.utils.config_from_keras_model(model, granularity='name')
  hls_config['Model']['Precision'] = 'ap_fixed<32,8>'
  hls_config['Model']['Strategy'] = 'Latency'
  for Layer in hls_config['LayerName'].keys():
      hls_config['LayerName'][Layer]['Precision'] = 'ap_fixed<32,8>'
      hls_config['LayerName'][Layer]['Strategy'] = 'Latency'

  print("-----------------------------------")
  print("Configuration")
  plotting.print_dict(hls_config)
  print("-----------------------------------")

  hls_model = hls4ml.converters.convert_from_keras_model(model, 
                                                         io_type='io_stream', 
                                                         backend='Vivado',
                                                         output_dir='{}'.format(model_name), 
                                                         project_name='{}'.format(model_name),
                                                         part='xcvu13p-fhga2104-1-e', 
                                                         hls_config=hls_config
                                                      )

  hls_model.compile()
  hls_model.build(csim=False, synth=True)

Expected behavior

I want to synthesize and get a two-input and one-output project directly.

Actual behavior

Error info:

  firmware/Ret_re5re6_in_box_con4_out.cpp: In function ‘void Ret_re5re6_in_box_con4_out(hls::stream<nnet::array<ap_fixed<32, 8>, 36> >&, hls::stream<nnet::array<ap_fixed<32, 8>, 36> >&, hls::stream<nnet::array<ap_fixed<32, 8>, 4> >&, hls::stream<nnet::array<ap_fixed<32, 8>, 4> >&)’:
  firmware/Ret_re5re6_in_box_con4_out.cpp:35:35: error: ‘layer4_t’ was not declared in this scope; did you mean ‘layer6_t’?
     35 |     nnet::concatenate2d<layer6_t, layer4_t, result_t, config5>(layer6_out, layer4_out, layer5_out); // concatenate_4
        |                                   ^~~~~~~~
        |                                   layer6_t
  firmware/Ret_re5re6_in_box_con4_out.cpp:35:76: error: ‘layer4_out’ was not declared in this scope; did you mean ‘layer6_out’?
     35 |     nnet::concatenate2d<layer6_t, layer4_t, result_t, config5>(layer6_out, layer4_out, layer5_out); // concatenate_4
        |                                                                            ^~~~~~~~~~
        |                                                                            layer6_out
  g++: error: Ret_re5re6_in_box_con4_out.o: No such file or directory
  Traceback (most recent call last):
    File "test_1.py", line 72, in <module>
      hls_model.compile()
    File "/home/hangfch/.conda/envs/hls4ml_38/lib/python3.8/site-packages/hls4ml/model/graph.py", line 664, in compile
      self._top_function_lib = ctypes.cdll.LoadLibrary(lib_name)
    File "/home/hangfch/.conda/envs/hls4ml_38/lib/python3.8/ctypes/__init__.py", line 451, in LoadLibrary
      return self._dlltype(name)
    File "/home/hangfch/.conda/envs/hls4ml_38/lib/python3.8/ctypes/__init__.py", line 373, in __init__
      self._handle = _dlopen(self._name, mode)
  OSError: Ret_re5re6_in_box_con4_out/firmware/Ret_re5re6_in_box_con4_out-eE62e9f6.so: cannot open shared object file: No such file or directory

Additional context

My whole project is here https://drive.google.com/file/d/1uXRbHjXYTA7SMLMF0FSddcJy0sFCnVQ_/view?usp=drive_link

calad0i commented 7 months ago

Confirmed. Will make a PR for this.

More details: replace_node method in ModelGraph did not consider the case when a parent/children node has more than 1 output/inputs.