Open technotip opened 2 months ago
vitoplantamura
commented
2 months ago hi,
The error indicates that you checked out the wrong commit of XNNPACK.
I suggest you to repeat all the steps, making sure to execute the "git checkout 1c8ee1b68f3a3e0847ec3c53c186c5909fa3fbd3" command.
Vito
technotip
commented
2 months ago Hi Vito,
Thank you ..I installed the XNNPACK as you said. I'm still getting the some errors.
Do I need particular branch for OnnxStream too?
Currently I'm just doing this: git clone https://github.com/vitoplantamura/OnnxStream.git
technotip
commented
2 months ago I'm getting the following error now:
cmake --build . --config Release
[ 33%] Building CXX object CMakeFiles/sd.dir/sd.cpp.o
/home/ubuntu/AI/OnnxStream/src/sd.cpp: In function ‘int main(int, char**)’:
/home/ubuntu/AI/OnnxStream/src/sd.cpp:1890:19: warning: format not a string literal and no format arguments [-Wformat-security]
1890 | printf(("Invalid command line argument: \"" + arg + "\".\n\n").c_str());
| ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/ubuntu/AI/OnnxStream/src/sd.cpp:1918:23: warning: format not a string literal and no format arguments [-Wformat-security]
1918 | printf(("Argument \"" + arg + "\" should be followed by a string.").c_str());
| ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[ 66%] Building CXX object CMakeFiles/sd.dir/onnxstream.cpp.o
In file included from /home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:1:
/home/ubuntu/AI/OnnxStream/src/onnxstream.h: In member function ‘onnxstream::tensor_vector<T> onnxstream::DiskPrefetchWeightsProvider::provide(const std::string&)’:
/home/ubuntu/AI/OnnxStream/src/onnxstream.h:521:22: error: ‘reverse’ is not a member of ‘std’
521 | std::reverse(m_entries.begin(), m_entries.end());
| ^~~~~~~
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp: In destructor ‘onnxstream::XnnPack::~XnnPack()’:
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:520:21: warning: ‘throw’ will always call ‘terminate’ [-Wterminate]
520 | throw std::runtime_error("failed to delete operator");
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:520:21: note: in C++11 destructors default to ‘noexcept’
In file included from /usr/include/c++/14/bits/stl_algobase.h:64,
from /usr/include/c++/14/vector:62,
from /home/ubuntu/AI/OnnxStream/src/onnxstream.h:4:
/usr/include/c++/14/bits/stl_pair.h: In instantiation of ‘constexpr std::pair<typename std::__strip_reference_wrapper<typename std::decay<_Tp>::type>::__type, typename std::__strip_reference_wrapper<typename std::decay<_Tp2>::type>::__type> std::make_pair(_T1&&, _T2&&) [with _T1 = float&; _T2 = float&; typename __strip_reference_wrapper<typename decay<_Tp>::type>::__type = float; typename decay<_Tp>::type = float; typename __strip_reference_wrapper<typename decay<_Tp2>::type>::__type = float; typename decay<_Tp2>::type = float]’:
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:3005:30: required from here
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:520:21: note: 3005 | return std::make_pair(prl_ctx.first, prl_ctx.second);
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:520:21: note: | ~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/14/bits/stl_pair.h:1116:5: note: parameter passing for argument of type ‘std::pair<float, float>’ when C++17 is enabled changed to match C++14 in GCC 10.1
1116 | make_pair(_T1&& __x, _T2&& __y)
| ^~~~~~~~~
gmake[2]: *** [CMakeFiles/sd.dir/build.make:90: CMakeFiles/sd.dir/onnxstream.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/sd.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2
could you try adding this line:
at the top of the onnxstream.h file?
Vito
technotip
commented
2 months ago Added, now I get the following errors:
$ cmake --build . --config Release
[ 33%] Building CXX object CMakeFiles/sd.dir/sd.cpp.o
/home/ubuntu/AI/OnnxStream/src/sd.cpp: In constructor ‘ncnn::Mat::Mat(int, int, int, void*)’:
/home/ubuntu/AI/OnnxStream/src/sd.cpp:122:13: error: ‘memcpy’ was not declared in this scope
122 | memcpy(v.data(), _data, total() * sizeof(float));
| ^~~~~~
/home/ubuntu/AI/OnnxStream/src/sd.cpp:41:1: note: ‘memcpy’ is defined in header ‘<cstring>’; this is probably fixable by adding ‘#include <cstring>’
40 | #include "onnxstream.h"
+++ |+#include <cstring>
41 | using namespace onnxstream;
/home/ubuntu/AI/OnnxStream/src/sd.cpp: In function ‘ncnn::Mat decoder_solver(ncnn::Mat&)’:
/home/ubuntu/AI/OnnxStream/src/sd.cpp:623:17: error: ‘memcpy’ was not declared in this scope
623 | memcpy((float*)res, model.m_data[0].get_vector<float>().data(), res.total() * sizeof(float));
| ^~~~~~
/home/ubuntu/AI/OnnxStream/src/sd.cpp:623:17: note: ‘memcpy’ is defined in header ‘<cstring>’; this is probably fixable by adding ‘#include <cstring>’
/home/ubuntu/AI/OnnxStream/src/sd.cpp: In lambda function:
/home/ubuntu/AI/OnnxStream/src/sd.cpp:762:13: error: ‘memcpy’ was not declared in this scope
762 | memcpy((float*)res, output_vec.data(), res.total() * sizeof(float));
| ^~~~~~
/home/ubuntu/AI/OnnxStream/src/sd.cpp:762:13: note: ‘memcpy’ is defined in header ‘<cstring>’; this is probably fixable by adding ‘#include <cstring>’
/home/ubuntu/AI/OnnxStream/src/sd.cpp: In function ‘ncnn::Mat prompt_solve(std::unordered_map<std::__cxx11::basic_string<char>, int>&, ncnn::Net&, std::string)’:
/home/ubuntu/AI/OnnxStream/src/sd.cpp:1156:21: error: ‘memcpy’ was not declared in this scope
1156 | memcpy((float*)res, model.m_data[0].get_vector<float>().data(), res.total() * sizeof(float));
| ^~~~~~
/home/ubuntu/AI/OnnxStream/src/sd.cpp:1156:21: note: ‘memcpy’ is defined in header ‘<cstring>’; this is probably fixable by adding ‘#include <cstring>’
/home/ubuntu/AI/OnnxStream/src/sd.cpp: In function ‘int main(int, char**)’:
/home/ubuntu/AI/OnnxStream/src/sd.cpp:1331:19: warning: format not a string literal and no format arguments [-Wformat-security]
1331 | printf(("Invalid command line argument: \"" + arg + "\".\n\n").c_str());
| ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/ubuntu/AI/OnnxStream/src/sd.cpp:1352:23: warning: format not a string literal and no format arguments [-Wformat-security]
1352 | printf(("Argument \"" + arg + "\" should be followed by a string.").c_str());
| ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/ubuntu/AI/OnnxStream/src/sd.cpp:1380:13: error: ‘memcpy’ was not declared in this scope
1380 | memcpy((float*)sample, vec.data(), sample.total() * sizeof(float));
| ^~~~~~
/home/ubuntu/AI/OnnxStream/src/sd.cpp:1380:13: note: ‘memcpy’ is defined in header ‘<cstring>’; this is probably fixable by adding ‘#include <cstring>’
gmake[2]: *** [CMakeFiles/sd.dir/build.make:76: CMakeFiles/sd.dir/sd.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/sd.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2
I did add #include <cstring> to sd.cpp previously, but still I got more errors. I'll add and post the errors in my next comment.
technotip
commented
2 months ago After adding #include <cstring> to sd.cpp I get huge list of errors. I'll post a part of the error message here:
$ cmake --build . --config Release
[ 33%] Building CXX object CMakeFiles/sd.dir/sd.cpp.o
/home/ubuntu/AI/OnnxStream/src/sd.cpp: In function ‘int main(int, char**)’:
/home/ubuntu/AI/OnnxStream/src/sd.cpp:1331:19: warning: format not a string literal and no format arguments [-Wformat-security]
1331 | printf(("Invalid command line argument: \"" + arg + "\".\n\n").c_str());
| ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/ubuntu/AI/OnnxStream/src/sd.cpp:1352:23: warning: format not a string literal and no format arguments [-Wformat-security]
1352 | printf(("Argument \"" + arg + "\" should be followed by a string.").c_str());
| ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[ 66%] Building CXX object CMakeFiles/sd.dir/onnxstream.cpp.o
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp: In member function ‘std::pair<std::vector<long unsigned int>, std::vector<T, onnxstream::Mallocator<T> > > onnxstream::XnnPack::softmax(std::vector<long unsigned int>&, T*, T*, Qu8SoftmaxData*)’:
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:1124:47: error: too many arguments to function ‘xnn_status xnn_create_softmax_nc_qu8(float, uint8_t, float, uint32_t, xnn_operator**)’
1124 | status = xnn_create_softmax_nc_qu8(
| ~~~~~~~~~~~~~~~~~~~~~~~~~^
1125 | channels /* channels */, channels /* input stride */, channels /* output stride */,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1126 | qu8_data->input_scale,
| ~~~~~~~~~~~~~~~~~~~~~~
1127 | qu8_data->output_zero_point,
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1128 | qu8_data->output_scale,
| ~~~~~~~~~~~~~~~~~~~~~~~
1129 | 0 /* flags */, &softmax_op);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:20:
/home/ubuntu/AI/XNNPACK/include/xnnpack.h:5831:17: note: declared here
5831 | enum xnn_status xnn_create_softmax_nc_qu8(
| ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp: In instantiation of ‘onnxstream::tensor_vector<T> onnxstream::XnnPack::convert(onnxstream::tensor_vector<T>&) [with U = short unsigned int; T = float; onnxstream::tensor_vector<T> = std::vector<float, onnxstream::Mallocator<float> >; onnxstream::tensor_vector<T> = std::vector<short unsigned int, onnxstream::Mallocator<short unsigned int> >]’:
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:1754:65: required from here
/home/ubuntu/AI/XNNPACK/include/xnnpack.h:5831:17: note: 1754 | auto data_fp32 = m_xnnpack->convert<uint16_t, float>(data);
/home/ubuntu/AI/XNNPACK/include/xnnpack.h:5831:17: note: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:84:45: error: invalid conversion from ‘xnn_status (*)(uint32_t, xnn_operator**)’ {aka ‘xnn_status (*)(unsigned int, xnn_operator**)’} to ‘xnn_status (*)(size_t, size_t, size_t, uint32_t, xnn_operator**)’ {aka ‘xnn_status (*)(long unsigned int, long unsigned int, long unsigned int, unsigned int, xnn_operator**)’} [-fpermissive]
84 | xnn_create_convert_nc_fxx_fxx = &xnn_create_convert_nc_f16_f32;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| xnn_status (*)(uint32_t, xnn_operator**) {aka xnn_status (*)(unsigned int, xnn_operator**)}
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:85:46: error: invalid conversion from ‘xnn_status (*)(xnn_operator_t, size_t, size_t, size_t, size_t, pthreadpool_t)’ {aka ‘xnn_status (*)(xnn_operator*, long unsigned int, long unsigned int, long unsigned int, long unsigned int, pthreadpool*)’} to ‘xnn_status (*)(xnn_operator_t, size_t, pthreadpool_t)’ {aka ‘xnn_status (*)(xnn_operator*, long unsigned int, pthreadpool*)’} [-fpermissive]
85 | xnn_reshape_convert_nc_fxx_fxx = &xnn_reshape_convert_nc_f16_f32;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| xnn_status (*)(xnn_operator_t, size_t, size_t, size_t, size_t, pthreadpool_t) {aka xnn_status (*)(xnn_operator*, long unsigned int, long unsigned int, long unsigned int, long unsigned int, pthreadpool*)}
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp: In instantiation of ‘onnxstream::tensor_vector<T> onnxstream::XnnPack::convert(onnxstream::tensor_vector<T>&) [with U = float; T = short unsigned int; onnxstream::tensor_vector<T> = std::vector<short unsigned int, onnxstream::Mallocator<short unsigned int> >; onnxstream::tensor_vector<T> = std::vector<float, onnxstream::Mallocator<float> >]’:
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:1763:65: required from here
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:85:46: error: 1763 | auto data_fp16 = m_xnnpack->convert<float, uint16_t>(data);
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:85:46: error: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:90:45: error: invalid conversion from ‘xnn_status (*)(uint32_t, xnn_operator**)’ {aka ‘xnn_status (*)(unsigned int, xnn_operator**)’} to ‘xnn_status (*)(size_t, size_t, size_t, uint32_t, xnn_operator**)’ {aka ‘xnn_status (*)(long unsigned int, long unsigned int, long unsigned int, unsigned int, xnn_operator**)’} [-fpermissive]
90 | xnn_create_convert_nc_fxx_fxx = &xnn_create_convert_nc_f32_f16;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| xnn_status (*)(uint32_t, xnn_operator**) {aka xnn_status (*)(unsigned int, xnn_operator**)}
/home/ubuntu/AI/OnnxStream/src/onnxstream.cpp:91:46: error: invalid conversion from ‘xnn_status (*)(xnn_operator_t, size_t, size_t, size_t, size_t, pthreadpool_t)’ {aka ‘xnn_status (*)(xnn_operator*, long unsigned int, long unsigned int, long unsigned int, long unsigned int, pthreadpool*)’} to ‘xnn_status (*)(xnn_operator_t, size_t, pthreadpool_t)’ {aka ‘xnn_status (*)(xnn_operator*, long unsigned int, pthreadpool*)’} [-fpermissive]
91 | xnn_reshape_convert_nc_fxx_fxx = &xnn_reshape_convert_nc_f32_f16;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| xnn_status (*)(xnn_operator_t, size_t, size_t, size_t, size_t, pthreadpool_t) {aka xnn_status (*)(xnn_operator*, long unsigned int, long unsigned int, long unsigned int, long unsigned int, pthreadpool*)}
This is the same error we were getting at the beginning. We are stuck.
Can you try with the Dockerfile? If you open the Dockerfile, at the beginning, there are the instructions to build the image.
Vito
technotip
commented
2 months ago Hi Vito,
But I'm running the same commands manually. Do you think running the dockerfile will make any difference? Btw, I'm using UBUNTU_VERSION=24.04 Server version, and not the desktop version.
vitoplantamura
commented
2 months ago It will most likely work. But you need to install Docker first.
Vito
Everything works fine until the cmake starts failing:
cmake --build . --config ReleaseI'm using: Linux ubuntu 6.1.0-1023-rockchip https://github.com/vitoplantamura/OnnxStream/issues/23-Ubuntu SMP Wed Aug 14 23:55:18 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
gcc version 13.2.0 (Ubuntu 13.2.0-23ubuntu4)
Device: Orange PI 5B 16GB Model
Updated gcc gcc (Ubuntu 14-20240412-0ubuntu1) 14.0.1 20240412 (experimental) [master r14-9935-g67e1433a94f]
Still the same errors.
The error message(s):