search

UCLA-VAST / tapa

TAPA is a dataflow HLS framework that features fast compilation, expressive programming model and generates high-frequency FPGA accelerators.
https://tapa.rtfd.io
MIT License
143 stars 27 forks source link

error: implicit instantiation of undefined template #127

Open enes1994 opened 1 year ago

enes1994 commented 1 year ago

Environment

Generating VecAdd.xilinx_u280_xdma_201920_3.hw.xo gives me the error below:

added vendor include path `/mnt/data/tools/Xilinx/Vitis_HLS/2021.1/include`
In file included from /mnt/data/work/sti/benchmark/xilinx/mv_pc28_u280/build/VecAdd.xilinx_u280_xdma_201920_3.hw.xo.tapa/flatten/flatten-e2bb650c-vadd.cpp:10:
In file included from /usr/include/tapa.h:6:
In file included from /usr/include/tapa/host/tapa.h:9:
/usr/include/tapa/host/task.h:163:15: error: implicit instantiation of undefined template 'tapa::internal::invoker<tapa::mmap<const ap_uint<512> > &>'
    internal::invoker<Func>::template invoke<Args...>(
              ^
/usr/include/tapa/host/task.h:138:12: note: in instantiation of function template specialization 'tapa::task::invoke<0, tapa::mmap<const ap_uint<512> > &, tapa::stream<ap_uint<512>,
      2> &, int &, int &, int &, 1>' requested here
    return invoke<join>(std::forward<Func>(func), "",
           ^
/mnt/data/work/sti/benchmark/xilinx/mv_pc28_u280/build/VecAdd.xilinx_u280_xdma_201920_3.hw.xo.tapa/flatten/flatten-e2bb650c-vadd.cpp:1598:4: note: in instantiation of function
      template specialization 'tapa::task::invoke<tapa::mmap<const ap_uint<512> > &, tapa::stream<ap_uint<512>, 2> &, int &, int &, int &>' requested here
  .invoke(
   ^
/usr/include/tapa/host/task.h:39:8: note: template is declared here
struct invoker;
       ^
E1118 21:02:01.576 tapa.steps.analyze:159] command ('/usr/bin/tapacc', '/mnt/data/work/sti/benchmark/xilinx/mv_pc28_u280/build/VecAdd.xilinx_u280_xdma_201920_3.hw.xo.tapa/flatten/flatten-e2bb650c-vadd.cpp', '-top', 'VecAdd', '--', '-std=c++17', '-I', '/home/sti/.local/lib/python3.8/site-packages/tapa/../../../src', '-isystem', '/home/sti/.local/lib/python3.8/site-packages/tapa/assets/clang', '-isystem', '/mnt/data/tools/Xilinx/Vitis_HLS/2021.1/include') failed with exit code -11
make[3]: *** [CMakeFiles/vadd-hw-xo.dir/build.make:61: VecAdd.xilinx_u280_xdma_201920_3.hw.xo] Error 245
make[2]: *** [CMakeFiles/Makefile2:211: CMakeFiles/vadd-hw-xo.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:191: CMakeFiles/vadd-hw.dir/rule] Error 2
make: *** [Makefile:157: vadd-hw] Error 2

This is an implementation of mine to create an effective matrix vector multiplication with TAPA with dataflow manner. Am I doing something wrong. I actually cant get what is the problem exactly.

Blaok commented 1 year ago

Can you check if the first argument to tapa::task::invoke() is the task function? I think it is a tapa::mmap right now.

enes1994 commented 1 year ago

I actually forgot to write the name of the function in one of invoke in device code. It is fine now. Thank you.

enes1994 commented 1 year ago

But now it gives me this error:

[ 66%] Building CXX object CMakeFiles/vadd.dir/host.cpp.o
[ 75%] Linking CXX executable vadd
/usr/bin/ld: src/libsrc.a(HlsMVexecutor.cpp.o): in function `HlsGeneralMVmultiplication<double>::exec_kernel(cl::Device const&, std::vector<double, JueStream::AlignedAllocator<double> > const&)':
/mnt/data/work/sti/benchmark/xilinx/mv_pc28_u280/src/HlsGeneralMVmultiplication.cpp:74: undefined reference to `VecAdd(tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double const>, tapa::mmap<double>, int, int, int)'
collect2: error: ld returned 1 exit status
make[3]: *** [CMakeFiles/vadd.dir/build.make:113: vadd] Error 1
make[2]: *** [CMakeFiles/Makefile2:238: CMakeFiles/vadd.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:191: CMakeFiles/vadd-hw.dir/rule] Error 2
make: *** [Makefile:157: vadd-hw] Error 2

I actually declared the function VecAdd like: 

void VecAdd(tapa::mmap<const double> array1,tapa::mmap<const double> array2,tapa::mmap<const double> array3,tapa::mmap<const double> array4,tapa::mmap<const double> array5,
        tapa::mmap<const double> array6,tapa::mmap<const double> array7,tapa::mmap<const double> array8,tapa::mmap<const double> array9,tapa::mmap<const double> array10,
        tapa::mmap<const double> array11,tapa::mmap<const double> array12,tapa::mmap<const double> array13,tapa::mmap<const double> array14,tapa::mmap<const double> array15,
        tapa::mmap<const double> array16,tapa::mmap<const double> array17,tapa::mmap<const double> array18,tapa::mmap<const double> array19,tapa::mmap<const double> array20,
        tapa::mmap<const double> array21,tapa::mmap<const double> array22,tapa::mmap<const double> array23,tapa::mmap<const double> array24,tapa::mmap<const double> array25,
        tapa::mmap<const double> array26,tapa::mmap<const double> array27,tapa::mmap<const double> array28,tapa::mmap<const double> arrayin, tapa::mmap<double> arrayout,int rows,int cols,
        int repeat);

And then I used it in a function like this:

template<typename T>
    std::vector<T, JueStream::AlignedAllocator<T>> HlsGeneralMVmultiplication<T>::exec_kernel(const cl::Device &device,const std::vector<T, JueStream::AlignedAllocator<T>> &input_vector) {

        std::filesystem::path bitstream (
        std::filesystem::path(BIN_DIR)/("vadd." + device.getInfo<CL_DEVICE_NAME>() + ".hw.xclbin"));

        this->output_vector.resize(rows_stride);
        // Compute
        // std::cout << "Compute " << std::endl;

        printf("start kernel\n");
        auto start = std::chrono::steady_clock::now(); 

        int64_t kernel_time_ns = tapa::invoke(
        VecAdd, bitstream.filename(),
        tapa::read_only_mmap<const double>(this->sub_matrices[0].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[1].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[2].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[3].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[4].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[5].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[6].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[7].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[8].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[9].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[10].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[11].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[12].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[13].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[14].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[15].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[16].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[17].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[18].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[19].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[20].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[21].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[22].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[23].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[24].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[25].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[26].data),
        tapa::read_only_mmap<const double>(this->sub_matrices[27].data),
        tapa::read_only_mmap<const double>(input_vector),
        tapa::write_only_mmap<double>(this->output_vector), 
        rows_stride,cols_stride,repeat_num);
        ......

decleration of VecAdd function and tapa::invoke is as I showed at up. They both are in the same cpp file. I am kind of stuck and cant find a solution. And that is quite urgent now for my thesis :S

Blaok commented 1 year ago

Did you link the kernel function with the host? This is required to run C simulation. If you don't want to link the kernel with the host, you can use the OpenCL APIs directly, or use the libfrt API (which is used to implement the TAPA invoke API).

enes1994 commented 1 year ago 
add_tapa_target(
  vadd-hw-xo
  INPUT ${CMAKE_CURRENT_SOURCE_DIR}/vadd.cpp
  TOP VecAdd
  CONNECTIVITY ${CMAKE_CURRENT_SOURCE_DIR}/connectivity.ini
  CONSTRAINT ${CMAKE_CURRENT_BINARY_DIR}/constraint.tcl
  PLATFORM ${PLATFORM})

Here is a a snipped code from my toplevel cmakelists.txt file. vadd.cpp is the device code. and I define VecAdd as the top level function in it. But I am calling tapa.invoke from a different file in a different directory other than where main function is. Is this the problem? If it is how can I fix it?

int64_t kernel_time_ns = tapa::invoke( VecAdd, bitstream.filename(), tapa::read_only_mmap(this->sub_matrices[0].data), tapa::read_only_mmap(this->sub_matrices[1].data), tapa::read_only_mmap(this->sub_matrices[2].data), tapa::read_only_mmap(this->sub_matrices[3].data), tapa::read_only_mmap(this->sub_matrices[4].data), tapa::read_only_mmap(this->sub_matrices[5].data), tapa::read_only_mmap(this->sub_matrices[6].data), tapa::read_only_mmap(this->sub_matrices[7].data), tapa::read_only_mmap(this->sub_matrices[8].data), tapa::read_only_mmap(this->sub_matrices[9].data), tapa::read_only_mmap(this->sub_matrices[10].data), tapa::read_only_mmap(this->sub_matrices[11].data), tapa::read_only_mmap(this->sub_matrices[12].data), tapa::read_only_mmap(this->sub_matrices[13].data), tapa::read_only_mmap(this->sub_matrices[14].data), tapa::read_only_mmap(this->sub_matrices[15].data), tapa::read_only_mmap(this->sub_matrices[16].data), tapa::read_only_mmap(this->sub_matrices[17].data), tapa::read_only_mmap(this->sub_matrices[18].data), tapa::read_only_mmap(this->sub_matrices[19].data), tapa::read_only_mmap(this->sub_matrices[20].data), tapa::read_only_mmap(this->sub_matrices[21].data), tapa::read_only_mmap(this->sub_matrices[22].data), tapa::read_only_mmap(this->sub_matrices[23].data), tapa::read_only_mmap(this->sub_matrices[24].data), tapa::read_only_mmap(this->sub_matrices[25].data), tapa::read_only_mmap(this->sub_matrices[26].data), tapa::read_only_mmap(this->sub_matrices[27].data), tapa::read_only_mmap(input_vector), tapa::write_only_mmap(this->output_vector), rows_stride,cols_stride,repeat_num);

Blaok commented 1 year ago

The add_tapa_target function only handles the .hw.xo target and is unrelated to the linking error. You need to make sure vadd.cpp is linked into the host executable. Would you mind posting the whole CMakeLists.txt?

enes1994 commented 1 year ago

This one is the top level cmake file:

cmake_minimum_required(VERSION 3.16.3)

set(CMAKE_CXX_STANDARD 17)
project(vadd)

set(PLATFORM
    xilinx_u280_xdma_201920_3
    CACHE STRING "Target FPGA platform")

set(TOP vadd)

set(CMAKE_CXX_FLAGS "${CMAKE_C_FLAGS} -Wno-write-strings")
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake") 

find_package(gflags REQUIRED)

find_package(Vitis)

include(${CMAKE_CURRENT_SOURCE_DIR}/tapa/cmake/apps.cmake)

include(${CMAKE_CURRENT_SOURCE_DIR}/tapa/cmake/TAPACCConfig.cmake)
include(${CMAKE_CURRENT_SOURCE_DIR}/tapa/cmake/FindSphinx.cmake)

add_subdirectory(src)

find_package(TAPA REQUIRED)
find_package(SDx REQUIRED)

add_executable(vadd)

target_sources(vadd PRIVATE host.cpp)

target_link_libraries(vadd PRIVATE ${TAPA} gflags src)
target_link_libraries(vadd PRIVATE tapa::tapa)

target_include_directories(src PRIVATE ${Vitis_INCLUDE_DIRS})
include(cmake/CMakeXilinx.cmake)

add_tapa_target(
  vadd-hw-xo
  INPUT ${CMAKE_CURRENT_SOURCE_DIR}/src/vadd.cpp
  TOP VecAdd
  CONNECTIVITY ${CMAKE_CURRENT_SOURCE_DIR}/connectivity.ini
  CONSTRAINT ${CMAKE_CURRENT_BINARY_DIR}/constraint.tcl
  PLATFORM ${PLATFORM})

add_xocc_hw_link_targets(
  ${CMAKE_CURRENT_BINARY_DIR}
  --config=${CMAKE_CURRENT_SOURCE_DIR}/connectivity.ini
  --vivado.synth.jobs 8
  --vivado.prop=run.impl_1.STEPS.PHYS_OPT_DESIGN.IS_ENABLED=1
  --vivado.prop=run.impl_1.STEPS.OPT_DESIGN.ARGS.DIRECTIVE=Explore
  --vivado.prop run.impl_1.STEPS.PLACE_DESIGN.ARGS.DIRECTIVE=EarlyBlockPlacement
  --vivado.prop=run.impl_1.STEPS.PHYS_OPT_DESIGN.ARGS.DIRECTIVE=Explore
  --vivado.prop run.impl_1.STEPS.ROUTE_DESIGN.ARGS.DIRECTIVE=Explore
  --vivado.prop=run.impl_1.STEPS.OPT_DESIGN.TCL.PRE=${CMAKE_CURRENT_BINARY_DIR}/constraint.tcl
  INPUT vadd-hw-xo
  HW_EMU_XCLBIN hw_emu_xclbin
  HW_XCLBIN hw_xclbin)

add_custom_target(
  vadd-cosim
  COMMAND $<TARGET_FILE:vadd>
  DEPENDS vadd ${hw_emu_xclbin}
  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})

add_custom_target(
  vadd-hw
  COMMAND $<TARGET_FILE:vadd>
  DEPENDS vadd ${hw_xclbin}
  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})

I actually tried to follow vadd example. I cant see what I am doing different than that. Only thing coming to my mind is instead of providing binary as: --bitstream=$<TARGET_PROPERTY:${hw_emu_xclbin},FILE_NAME> as an argument to main function, I am trying to read it with: 

std::filesystem::path bitstream (
std::filesystem::path(BIN_DIR)/("vadd." + device.getInfo<CL_DEVICE_NAME>() + ".hw.xclbin"));

then pass "bitstream" as an argument to tapa::invoke as follows:

int64_t kernel_time_ns = tapa::invoke( VecAdd, bitstream.filename(), tapa::read_only_mmap(this->sub_matrices[0].data), tapa::read_only_mmap(this->sub_matrices[1].data), tapa::read_only_mmap(this->sub_matrices[2].data), tapa::read_only_mmap(this->sub_matrices[3].data), tapa::read_only_mmap(this->sub_matrices[4].data), tapa::read_only_mmap(this->sub_matrices[5].data), tapa::read_only_mmap(this->sub_matrices[6].data), tapa::read_only_mmap(this->sub_matrices[7].data), tapa::read_only_mmap(this->sub_matrices[8].data), tapa::read_only_mmap(this->sub_matrices[9].data), tapa::read_only_mmap(this->sub_matrices[10].data), tapa::read_only_mmap(this->sub_matrices[11].data), tapa::read_only_mmap(this->sub_matrices[12].data), tapa::read_only_mmap(this->sub_matrices[13].data), tapa::read_only_mmap(this->sub_matrices[14].data), tapa::read_only_mmap(this->sub_matrices[15].data), tapa::read_only_mmap(this->sub_matrices[16].data), tapa::read_only_mmap(this->sub_matrices[17].data), tapa::read_only_mmap(this->sub_matrices[18].data), tapa::read_only_mmap(this->sub_matrices[19].data), tapa::read_only_mmap(this->sub_matrices[20].data), tapa::read_only_mmap(this->sub_matrices[21].data), tapa::read_only_mmap(this->sub_matrices[22].data), tapa::read_only_mmap(this->sub_matrices[23].data), tapa::read_only_mmap(this->sub_matrices[24].data), tapa::read_only_mmap(this->sub_matrices[25].data), tapa::read_only_mmap(this->sub_matrices[26].data), tapa::read_only_mmap(this->sub_matrices[27].data), tapa::read_only_mmap(input_vector), tapa::write_only_mmap(this->output_vector), rows_stride,cols_stride,repeat_num);

Is this the source to the main problem?

Blaok commented 1 year ago

target_sources(vadd PRIVATE host.cpp)

This line needs to be

target_sources(vadd PRIVATE vadd.cpp host.cpp)

enes1994 commented 1 year ago

it is unfortunately still the same. Undefined reference error. Still missing sth. I really cant figure it out. also during the sythesis I keep getting failed to get a valid floorplanning. Arguments are seen as read and write both even if I specify them as either read only or write only as follows:

function(add_tapa_target target_name)
  # Generate Xilinx object file from TAPA C++.
  #
  # The added target will have the following properties:
  #
  # * FILE_NAME
  # * TARGET
  # * KERNEL
  # * PLATFORM
  #
  # Positional Arguments:
  #
  # * target_name: Name of the added cmake target.
  #
  # Required Named Arguments:
  #
  # * INPUT: Input filename.
  # * TOP: Name of the top-level task, which will be used the kernel name.
  # * PLATFORM: SDAccel platform name or path.
  #
  # Optional Named Arguments:
  #
  # * OUTPUT: Optional, output filename, default to ${TOP}.${PLATFORM}.hw.xo
  # * TAPA_CLI: Optional, path to the tapa executable.
  # * TAPACC: Optional, path to the tapacc executable.
  # * TAPA_CLANG: Optional, path to the tapa-clang executable.
  # * CFLAGS: Optional, cflags for kernel, space separated.
  # * FRT_INTERFACE: Optional, output FRT interface filename.
  # * CLOCK_PERIOD: Optional, override the clock period.
  # * PART_NUM: Optional, override the part number.
  # * DIRECTIVE: Optional, apply partitioning directives.
  # * CONNECTIVITY: Optional, specifies DRAM connections using Xilinx's ini.
  # * CONSTRAINT: Optional if DIRECTIVE is not set, generate partitioning
  #   constraints.
  cmake_parse_arguments(
    TAPA
    ""
    "OUTPUT;INPUT;TOP;PLATFORM;TAPA_CLI;TAPACC;TAPA_CLANG;CFLAGS;FRT_INTERFACE;CLOCK_PERIOD;PART_NUM;DIRECTIVE;CONNECTIVITY;CONSTRAINT"
    ""
    ${ARGN})
  if(NOT TAPA_INPUT)
    message(FATAL_ERROR "INPUT not specified")
  endif()
  if(NOT TAPA_PLATFORM)
    message(FATAL_ERROR "PLATFORM not specified")
  endif()
  if(NOT TAPA_TOP)
    message(FATAL_ERROR "TOP not specified")
  endif()

  if(NOT TAPA_OUTPUT)
    set(TAPA_OUTPUT
        ${CMAKE_CURRENT_BINARY_DIR}/${TAPA_TOP}.${TAPA_PLATFORM}.hw.xo)
  endif()
  get_filename_component(TAPA_INPUT ${TAPA_INPUT} ABSOLUTE)
  get_filename_component(TAPA_OUTPUT ${TAPA_OUTPUT} ABSOLUTE)

  if(TAPA_TAPA_CLI)
    set(TAPA_CLI ${TAPA_TAPA_CLI})
  endif()
  if(NOT TAPA_CLI)
    find_program(TAPA_CLI tapa PATHS $ENV{HOME}/.local/bin)
  endif()
  if(NOT TAPA_CLI)
    message(FATAL_ERROR "cannot find tapa")
  endif()

  if(TAPA_TAPACC)
    set(TAPACC ${TAPA_TAPACC})
  endif()
  if(NOT TAPACC)
    find_program(TAPACC tapacc)
  endif()
  if(NOT TAPACC)
    message(FATAL_ERROR "cannot find tapacc")
  endif()

  if(TAPA_TAPA_CLANG)
    set(TAPA_CLANG ${TAPA_TAPA_CLANG})
  endif()
  if(NOT TAPA_CLANG)
    find_program(TAPA_CLANG tapa-clang)
  endif()
  if(NOT TAPA_CLANG)
    message(FATAL_ERROR "cannot find tapa-clang")
  endif()
  set(A0 "A0")
  set(A1 "A1")
  set(A2 "A2")
  set(A3 "A3")
  set(A4 "A4")
  set(A5 "A5")
  set(A6 "A6")
  set(A7 "A7")
  set(A8 "A8")
  set(A9 "A9")
  set(A10 "A10")
  set(A11 "A11")
  set(A12 "A12")
  set(A13 "A13")
  set(A14 "A14")
  set(A15 "A15")
  set(A16 "A16")
  set(A17 "A17")
  set(A18 "A18")
  set(A19 "A19")
  set(A20 "A20")
  set(A21 "A21")
  set(A22 "A22")
  set(A23 "A23")
  set(A24 "A24")
  set(A25 "A25")
  set(A26 "A26")
  set(A27 "A27")
  set(x "x")
  set(y "y")
  set(arg0 --read-only-args ${A0})
  set(arg1 --read-only-args ${A1})
  set(arg2 --read-only-args ${A2})
  set(arg3 --read-only-args ${A3})
  set(arg4 --read-only-args ${A4})
  set(arg5 --read-only-args ${A5})
  set(arg6 --read-only-args ${A6})
  set(arg7 --read-only-args ${A7})
  set(arg8 --read-only-args ${A8})
  set(arg9 --read-only-args ${A9})
  set(arg10 --read-only-args ${A10})
  set(arg11 --read-only-args ${A11})
  set(arg12 --read-only-args ${A12})
  set(arg13 --read-only-args ${A13})
  set(arg14 --read-only-args ${A14})
  set(arg15 --read-only-args ${A15})
  set(arg16 --read-only-args ${A16})
  set(arg17 --read-only-args ${A17})
  set(arg18 --read-only-args ${A18})
  set(arg19 --read-only-args ${A19})
  set(arg20 --read-only-args ${A20})
  set(arg21 --read-only-args ${A21})
  set(arg22 --read-only-args ${A22})
  set(arg23 --read-only-args ${A23})
  set(arg24 --read-only-args ${A24})
  set(arg25 --read-only-args ${A25})
  set(arg26 --read-only-args ${A26})
  set(arg27 --read-only-args ${A27})
  set(arg29 --read-only-args ${x})
  set(arg30 --write-only-args ${y})

  set(tapa_cmd ${TAPA_CLI} --work-dir ${TAPA_OUTPUT}.tapa)
  list(APPEND tapa_cmd analyze)
  list(APPEND tapa_cmd -f ${TAPA_INPUT})
  list(APPEND tapa_cmd --top ${TAPA_TOP})
  list(APPEND tapa_cmd --tapacc ${TAPACC})
  list(APPEND tapa_cmd --tapa-clang ${TAPA_CLANG})
  if(TAPA_CFLAGS)
    list(APPEND tapa_cmd --cflags ${TAPA_CFLAGS})
  endif()

  list(APPEND tapa_cmd synth)
  list(APPEND tapa_cmd --platform ${TAPA_PLATFORM})
  if(TAPA_CLOCK_PERIOD)
    list(APPEND tapa_cmd --clock-period ${TAPA_CLOCK_PERIOD})
  endif()
  if(TAPA_PART_NUM)
    list(APPEND tapa_cmd --part-num ${TAPA_PART_NUM})
  endif()

  if(TAPA_CONNECTIVITY AND TAPA_CONSTRAINT)
    list(APPEND tapa_cmd optimize-floorplan)
    list(APPEND tapa_cmd --connectivity ${TAPA_CONNECTIVITY})
  endif()
  list(APPEND tapa_cmd ${arg0})
  list(APPEND tapa_cmd ${arg1})
  list(APPEND tapa_cmd ${arg2})
  list(APPEND tapa_cmd ${arg3})
  list(APPEND tapa_cmd ${arg4})
  list(APPEND tapa_cmd ${arg5})
  list(APPEND tapa_cmd ${arg6})
  list(APPEND tapa_cmd ${arg7})
  list(APPEND tapa_cmd ${arg8})
  list(APPEND tapa_cmd ${arg9})
  list(APPEND tapa_cmd ${arg10})
  list(APPEND tapa_cmd ${arg11})
  list(APPEND tapa_cmd ${arg12})
  list(APPEND tapa_cmd ${arg13})
  list(APPEND tapa_cmd ${arg14})
  list(APPEND tapa_cmd ${arg15})
  list(APPEND tapa_cmd ${arg16})
  list(APPEND tapa_cmd ${arg17})
  list(APPEND tapa_cmd ${arg18})
  list(APPEND tapa_cmd ${arg19})
  list(APPEND tapa_cmd ${arg20})
  list(APPEND tapa_cmd ${arg21})
  list(APPEND tapa_cmd ${arg22})
  list(APPEND tapa_cmd ${arg23})
  list(APPEND tapa_cmd ${arg24})
  list(APPEND tapa_cmd ${arg25})
  list(APPEND tapa_cmd ${arg26})
  list(APPEND tapa_cmd ${arg27})
  list(APPEND tapa_cmd ${arg28})
  list(APPEND tapa_cmd ${arg29})
  list(APPEND tapa_cmd ${arg30})

  # If we have to put unparsed arguments somewhere, this is the best place; most
  # additional arguments are for `tapa synth` and `tapa optimize-floorplan`.
  list(APPEND tapa_cmd ${TAPA_UNPARSED_ARGUMENTS})

  list(APPEND tapa_cmd link)
  if(TAPA_CONNECTIVITY AND TAPA_CONSTRAINT)
    list(APPEND tapa_cmd --floorplan-output ${TAPA_CONSTRAINT})
  endif()

  list(APPEND tapa_cmd pack)
  list(APPEND tapa_cmd --output ${TAPA_OUTPUT})

  add_custom_command(
    OUTPUT ${TAPA_OUTPUT}
    COMMAND ${tapa_cmd}
    DEPENDS ${TAPA_INPUT}
    VERBATIM)

  add_custom_target(${target_name} DEPENDS ${TAPA_OUTPUT})
  set_target_properties(
    ${target_name}
    PROPERTIES FILE_NAME ${TAPA_OUTPUT} TARGET hw KERNEL ${TAPA_TOP}
               PLATFORM ${TAPA_PLATFORM} DRAM_MAPPING "")
endfunction()

I have changed TAPACCConfig file to fit my problem.

int64_t kernel_time_ns = tapa::invoke( VecAdd, bitstream.filename(), tapa::read_only_mmap(this->sub_matrices[0].data), tapa::read_only_mmap(this->sub_matrices[1].data), tapa::read_only_mmap(this->sub_matrices[2].data), tapa::read_only_mmap(this->sub_matrices[3].data), tapa::read_only_mmap(this->sub_matrices[4].data), tapa::read_only_mmap(this->sub_matrices[5].data), tapa::read_only_mmap(this->sub_matrices[6].data), tapa::read_only_mmap(this->sub_matrices[7].data), tapa::read_only_mmap(this->sub_matrices[8].data), tapa::read_only_mmap(this->sub_matrices[9].data), tapa::read_only_mmap(this->sub_matrices[10].data), tapa::read_only_mmap(this->sub_matrices[11].data), tapa::read_only_mmap(this->sub_matrices[12].data), tapa::read_only_mmap(this->sub_matrices[13].data), tapa::read_only_mmap(this->sub_matrices[14].data), tapa::read_only_mmap(this->sub_matrices[15].data), tapa::read_only_mmap(this->sub_matrices[16].data), tapa::read_only_mmap(this->sub_matrices[17].data), tapa::read_only_mmap(this->sub_matrices[18].data), tapa::read_only_mmap(this->sub_matrices[19].data), tapa::read_only_mmap(this->sub_matrices[20].data), tapa::read_only_mmap(this->sub_matrices[21].data), tapa::read_only_mmap(this->sub_matrices[22].data), tapa::read_only_mmap(this->sub_matrices[23].data), tapa::read_only_mmap(this->sub_matrices[24].data), tapa::read_only_mmap(this->sub_matrices[25].data), tapa::read_only_mmap(this->sub_matrices[26].data), tapa::read_only_mmap(this->sub_matrices[27].data), tapa::read_only_mmap(input_vector), tapa::write_only_mmap(this->output_vector), rows_stride,cols_stride,repeat_num);

you can see here also I specify them as read_only or write only. As I understood tapa_cmd is tapa compiler. But cant set the arguments as read or write only. Here is top level function in device code:

void VecAdd(
    tapa::mmap<const ap_uint<512>> A0, 
    tapa::mmap<const ap_uint<512>> A1, 
    tapa::mmap<const ap_uint<512>> A2, 
    tapa::mmap<const ap_uint<512>> A3, 
    tapa::mmap<const ap_uint<512>> A4, 
    tapa::mmap<const ap_uint<512>> A5, 
    tapa::mmap<const ap_uint<512>> A6, 
    tapa::mmap<const ap_uint<512>> A7, 
    tapa::mmap<const ap_uint<512>> A8, 
    tapa::mmap<const ap_uint<512>> A9, 
    tapa::mmap<const ap_uint<512>> A10, 
    tapa::mmap<const ap_uint<512>> A11, 
    tapa::mmap<const ap_uint<512>> A12, 
    tapa::mmap<const ap_uint<512>> A13, 
    tapa::mmap<const ap_uint<512>> A14, 
    tapa::mmap<const ap_uint<512>> A15, 
    tapa::mmap<const ap_uint<512>> A16, 
    tapa::mmap<const ap_uint<512>> A17, 
    tapa::mmap<const ap_uint<512>> A18, 
    tapa::mmap<const ap_uint<512>> A19, 
    tapa::mmap<const ap_uint<512>> A20, 
    tapa::mmap<const ap_uint<512>> A21, 
    tapa::mmap<const ap_uint<512>> A22, 
    tapa::mmap<const ap_uint<512>> A23, 
    tapa::mmap<const ap_uint<512>> A24, 
    tapa::mmap<const ap_uint<512>> A25, 
    tapa::mmap<const ap_uint<512>> A26, 
    tapa::mmap<const ap_uint<512>> A27, 
    tapa::mmap<const ap_uint<512>> x,
    tapa::mmap<ap_uint<512>> y,
    int rows_stride,
    int cols_stride,
    int repeat_num
) {
Blaok commented 1 year ago

it is unfortunately still the same. Undefined reference error.

That doesn't make any sense. Did you put the definition and declaration in different linkages, i.e., did you use extern "C"? If so, can you check if you are using it consistently for both the definition and declaration?

I have changed TAPACCConfig file to fit my problem.

Changes to TAPACCConfig.cmake won't be effective unless you build TAPA from source. If you just want to specify additional arguments to the TAPA target, please do the following:

add_tapa_target(
  vadd-hw-xo
  --read-only-args=A
  --read-only-args=x
  --write-only-args=y
  INPUT ${CMAKE_CURRENT_SOURCE_DIR}/vadd.cpp
  TOP VecAdd
  CONNECTIVITY ${CMAKE_CURRENT_SOURCE_DIR}/connectivity.ini
  CONSTRAINT ${CMAKE_CURRENT_BINARY_DIR}/constraint.tcl
  PLATFORM ${PLATFORM})

Note that this requries TAPA 0.0.20221113.1.

enes1994 commented 1 year ago

Thank you very much @Blaok . There is another issue while I am running the Floorplanning. Here is .ini file I am using:


[connectivity]

sp=VecAdd.x:HBM[16]
sp=VecAdd.y:HBM[17]
sp=VecAdd.A0:HBM[0]
sp=VecAdd.A1:HBM[1]
sp=VecAdd.A2:HBM[2]
sp=VecAdd.A3:HBM[3]
sp=VecAdd.A4:HBM[4]
sp=VecAdd.A5:HBM[5]
sp=VecAdd.A6:HBM[6]
sp=VecAdd.A7:HBM[7]
sp=VecAdd.A8:HBM[8]
sp=VecAdd.A9:HBM[9]
sp=VecAdd.A10:HBM[10]
sp=VecAdd.A11:HBM[11]
sp=VecAdd.A12:HBM[12]
sp=VecAdd.A13:HBM[13]
sp=VecAdd.A14:HBM[14]
sp=VecAdd.A15:HBM[15]

A0 to A15 is actually the matrix splitted to 16 HBMs. and x is the input and y is the output vector. 

W1123 10:04:45.539 tapa.floorplan:263] failed to get a valid floorplanning
I1123 10:04:45.539 tapa.steps.common:92] writing TAPA settings to json `/mnt/data/work/sti/benchmark/xilinx/mv_pc28_u280/build/VecAdd.xilinx_u280_xdma_201920_3.hw.xo.tapa/settings.json`.
I1123 10:04:45.540 tapa.steps.common:46] reusing TAPA settings from upstream flows.
I1123 10:04:45.540 tapa.floorplan:77] generate the floorplan constraint at /mnt/data/work/sti/benchmark/xilinx/mv_pc28_u280/build/constraint.tcl
W1123 10:04:45.540 tapa.core:391] generate top rtl without floorplanning

I cant understand why it can not find a suitable floorplan. I dont have Gurobi solver and it shows some warnings. Is it related to that or because the arguments to top function are all read and write type?

Blaok commented 1 year ago

I cant understand why it can not find a suitable floorplan.

My personal experience is that floorplanning can be very tricky. There are something you could try, but it's hard to say what would work. Two common reasons for floorplanning failures: 1) Some tasks are too large to fit in a single slot. It is generally helpful to break large tasks into smaller ones; 2) The total resource usage is too high. There is no good solution if this is the case, unless you can reduce the resource usage.

I dont have Gurobi solver and it shows some warnings. Is it related to that or because the arguments to top function are all read and write type?

I'm not seeing enough evidence regarding whether Gurobi solver would be helpful, but I'm pretty sure specifying --read/write-only-args won't be harmful (unless they are incorrect).

Licheng-Guo commented 1 year ago

The log will print out in which file the detailed debug messages are saved, could you take a look at that file? autobridge-[timestamp].log