search

digitalbrain79 / darknet-nnpack

Darknet with NNPACK
Other
305 stars 75 forks source link

NNPACK build issue - ninja #53

Open mipsan opened 4 years ago

mipsan commented 4 years ago

Dear,

during $NINJA_HOME/ninja running, following error occured and build process stopped;

[30/147] CC deps/pthreadpool/src/pthreads.c
FAILED: /home/pi/Downloads/NNPACK-darknet/build/deps/pthreadpool/src/pthreads.c.o 
gcc -o /home/pi/Downloads/NNPACK-darknet/build/deps/pthreadpool/src/pthreads.c.o -c /home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/src/pthreads.c -MMD -MF /home/pi/Downloads/NNPACK-darknet/build/deps/pthreadpool/src/pthreads.c.o.d -std=gnu11 -g -pthread -O2    -I/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include -I/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/src -I/home/pi/Downloads/NNPACK-darknet/deps/fxdiv/include
In file included from /home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:50:
/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include/pthreadpool.h:727:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include/pthreadpool.h:733:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include/pthreadpool.h:740:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include/pthreadpool.h:747:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include/pthreadpool.h:756:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include/pthreadpool.h:767:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/cccFH11x.s: Assembler messages:
/tmp/cccFH11x.s:86: Error: selected processor does not support `yield' in ARM mode
/tmp/cccFH11x.s:420: Error: selected processor does not support `yield' in ARM mode

I use RPi 4 and official BusterOS. During instructions, python3 is used instead of python2 here: python ./configure.py --backend auto

Any help about this please?

updated:

Installed confu is 0.0.1 version. Seems very old one.

angmorpri commented 4 years ago

I also faced that error on my Raspberry Pi 3. It seems that it has to do with the last update of Maratyszcza/pthreadpool repo, which is downloaded sometime during the NNPACK build.

I solved it just by changing the commit of the pthreadpool repo that is cloned; but I have to say that I do not use the confu method to build NNPACK. Instead -as recommended in Maratyszcza/NNPACK-, I use the cmake rules. In this case, I just had to go to NNPACK/cmake/DownloadPThreadPool.cmake and change the line GIT_TAG master to GIT_TAG 5690b5ceada160444a916d31ef72e381f5e52d67, being the new tag the last commit I was able to get working.

The rules I follow for building NNPACK are:

cd NNPACK
mkdir build && cd build
cmake -G Ninja ..
ninja
sudo ninja install
mipsan commented 4 years ago

Thanks @angmorpri for your reply.

I made a mistake, my failure was related to darknet-nnpack. It was totally my fault.

Using NNPACK's cmake resources, I did build NNPACK-darknet. In fact, some modifications in CMakeLists.txt.

Thanks,

updated: BUILD step passed but not working with 'NNPACK=1' option. nothing detected. I have to dig what's wrong from the first step again.

Thanks,

mipsan commented 4 years ago

I tested 2 cases:

  1. use NNPACk instead of NNPACK-darknet: darknet-nnpack does not show any detected objects. It shows video frames and FPS info. build and copy some libraries info system directory.

  2. NNPACK-darknet: copy cmake files from NNPACK and modify somewhere to build.

    • pthreadpool GIT_TAG
    • CMakeLists.txt: some files are missed This does not show any result as well. Just video frames and FPS info.

Both of them, AVG_FPS is approx. 3.4 with YOLOv3. mipi camera is used. If I use YOLOV3_tiny, it increases up to 11 fps.

Problem is that it 'cannot detect' at all. Any ideas please? Thanks,

HaroldSP commented 4 years ago

Dear,

during $NINJA_HOME/ninja running, following error occured and build process stopped;

[30/147] CC deps/pthreadpool/src/pthreads.c
FAILED: /home/pi/Downloads/NNPACK-darknet/build/deps/pthreadpool/src/pthreads.c.o 
gcc -o /home/pi/Downloads/NNPACK-darknet/build/deps/pthreadpool/src/pthreads.c.o -c /home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/src/pthreads.c -MMD -MF /home/pi/Downloads/NNPACK-darknet/build/deps/pthreadpool/src/pthreads.c.o.d -std=gnu11 -g -pthread -O2    -I/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include -I/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/src -I/home/pi/Downloads/NNPACK-darknet/deps/fxdiv/include
In file included from /home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:50:
/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include/pthreadpool.h:727:2: warning: ‘pthreadpool_function_1d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include/pthreadpool.h:733:2: warning: ‘pthreadpool_function_1d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_1d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include/pthreadpool.h:740:2: warning: ‘pthreadpool_function_2d_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include/pthreadpool.h:747:2: warning: ‘pthreadpool_function_2d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_2d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include/pthreadpool.h:756:2: warning: ‘pthreadpool_function_3d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_3d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/pi/Downloads/NNPACK-darknet/deps/pthreadpool/include/pthreadpool.h:767:2: warning: ‘pthreadpool_function_4d_tiled_t’ is deprecated [-Wdeprecated-declarations]
  pthreadpool_function_4d_tiled_t function,
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/tmp/cccFH11x.s: Assembler messages:
/tmp/cccFH11x.s:86: Error: selected processor does not support `yield' in ARM mode
/tmp/cccFH11x.s:420: Error: selected processor does not support `yield' in ARM mode

I use RPi 4 and official BusterOS. During instructions, python3 is used instead of python2 here: python ./configure.py --backend auto

Any help about this please?

updated:

Installed confu is 0.0.1 version. Seems very old one.

Having the same errors and no idea how to fix them:

/tmp/cccFH11x.s: Assembler messages: /tmp/cccFH11x.s:86: Error: selected processor does not support yield' in ARM mode /tmp/cccFH11x.s:420: Error: selected processor does not supportyield' in ARM mode

mipsan commented 4 years ago

@HaroldSP,

I could not run darknet-nnpack yet. built but not working. detection box count is not valid while running 'darknet detector demo ...'

TO test, I used NNPACK's CMakeLists.txt and cmake directory. I did some modification for slightly different file lists in CMakeLists.txt according to NNPACK-darknet/src lists.

I will dig it what's wrong. thanks,

angmorpri commented 4 years ago

@HaroldSP , check that issue on original NNPACK repo, it may help you.

@mipsan I'm sorry but I have not tested YOLO with camera stream, so I cannot help you. Anyway, as it something different from the original issue, you may close this one if that is solved, and open a new one with your current problem.

HaroldSP commented 4 years ago

@mipsan @angmorpri , thanks, it actually helped to build, but later than I was following the various instructions, like:

do a ls and you should be able to find the folders lib and include if all went well: ls Test if NNPACK is working: bin/convolution-inference-smoketest

and: 

sudo cp -a lib/* /usr/lib/
sudo cp include/nnpack.h /usr/include/
sudo cp deps/pthreadpool/include/pthreadpool.h /usr/include/

And I couldn't find any folder like lib. And it was not working afterwards. To be honest, it is all quite confusing with no clear instruction.

angmorpri commented 4 years ago

@HaroldSP those instructions may only be followed if you are using confu, which is a depcrated method. With cmake you only need the steps I wrote in the first comment.

HaroldSP commented 4 years ago

@mipsan @angmorpri

Thanks everyone, finally it works! Here's the small wiki that I've written.

saifddine commented 4 years ago

Hello, i tried all the proposed solutions and i still can't run this on my RPi 3 B, i get the same errors even though i tried this so many times, i'd really appreciate any help [1/61] LINK bin/convolution-benchmark FAILED: /home/pi/ninja/NNPACK-darknet/bin/convolution-benchmark g++ -pthread -o /home/pi/ninja/NNPACK-darknet/bin/convolution-benchmark /home/pi/ninja/NNPACK-darknet/build/bench/convolution.c.o /home/pi/ninja/NNPACK-darknet/build/bench/median.c.o /home/pi/ninja/NNPACK-darknet/build/bench/memread.c.o -lrt /home/pi/ninja/NNPACK-darknet/lib/libnnpack.a /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in functionpthreadpool_create': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:258: undefined reference to pthreadpool_allocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in functionpthreadpool_destroy': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference to pthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference topthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/bin/convolution-benchmark: internal symbol pthreadpool_deallocate' isn't defined /usr/bin/ld: final link failed: bad value collect2: error: ld returned 1 exit status [2/61] LINK bin/relu-benchmark FAILED: /home/pi/ninja/NNPACK-darknet/bin/relu-benchmark g++ -pthread -o /home/pi/ninja/NNPACK-darknet/bin/relu-benchmark /home/pi/ninja/NNPACK-darknet/build/bench/relu.c.o /home/pi/ninja/NNPACK-darknet/build/bench/median.c.o /home/pi/ninja/NNPACK-darknet/build/bench/memread.c.o -lrt /home/pi/ninja/NNPACK-darknet/lib/libnnpack.a /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in functionpthreadpool_create': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:258: undefined reference to pthreadpool_allocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in functionpthreadpool_destroy': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference to pthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference topthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/bin/relu-benchmark: internal symbol pthreadpool_deallocate' isn't defined /usr/bin/ld: final link failed: bad value collect2: error: ld returned 1 exit status [3/61] LINK bin/pooling-benchmark FAILED: /home/pi/ninja/NNPACK-darknet/bin/pooling-benchmark g++ -pthread -o /home/pi/ninja/NNPACK-darknet/bin/pooling-benchmark /home/pi/ninja/NNPACK-darknet/build/bench/pooling.c.o /home/pi/ninja/NNPACK-darknet/build/bench/median.c.o /home/pi/ninja/NNPACK-darknet/build/bench/memread.c.o -lrt /home/pi/ninja/NNPACK-darknet/lib/libnnpack.a /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in functionpthreadpool_create': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:258: undefined reference to pthreadpool_allocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in functionpthreadpool_destroy': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference to pthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference topthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/bin/pooling-benchmark: internal symbol pthreadpool_deallocate' isn't defined /usr/bin/ld: final link failed: bad value collect2: error: ld returned 1 exit status [4/61] LINK bin/fully-connected-benchmark FAILED: /home/pi/ninja/NNPACK-darknet/bin/fully-connected-benchmark g++ -pthread -o /home/pi/ninja/NNPACK-darknet/bin/fully-connected-benchmark /home/pi/ninja/NNPACK-darknet/build/bench/fully-connected.c.o /home/pi/ninja/NNPACK-darknet/build/bench/median.c.o /home/pi/ninja/NNPACK-darknet/build/bench/memread.c.o -lrt /home/pi/ninja/NNPACK-darknet/lib/libnnpack.a /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in functionpthreadpool_create': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:258: undefined reference to pthreadpool_allocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in functionpthreadpool_destroy': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference to pthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference topthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/bin/fully-connected-benchmark: internal symbol pthreadpool_deallocate' isn't defined /usr/bin/ld: final link failed: bad value collect2: error: ld returned 1 exit status [5/61] LINK bin/transform-benchmark FAILED: /home/pi/ninja/NNPACK-darknet/bin/transform-benchmark g++ -pthread -o /home/pi/ninja/NNPACK-darknet/bin/transform-benchmark /home/pi/ninja/NNPACK-darknet/build/bench/transform.c.o /home/pi/ninja/NNPACK-darknet/build/bench/median.c.o /home/pi/ninja/NNPACK-darknet/build/bench/memread.c.o -lrt /home/pi/ninja/NNPACK-darknet/lib/libnnpack.a /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in functionpthreadpool_create': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:258: undefined reference to pthreadpool_allocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in functionpthreadpool_destroy': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference to pthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference topthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/bin/transform-benchmark: internal symbol pthreadpool_deallocate' isn't defined /usr/bin/ld: final link failed: bad value collect2: error: ld returned 1 exit status [6/61] LINK bin/convolution-output-alexnet-test FAILED: /home/pi/ninja/NNPACK-darknet/bin/convolution-output-alexnet-test g++ -pthread -o /home/pi/ninja/NNPACK-darknet/bin/convolution-output-alexnet-test /home/pi/ninja/NNPACK-darknet/build/src/ref/convolution-output.c.o /home/pi/ninja/NNPACK-darknet/build/src/ref/convolution-input-gradient.c.o /home/pi/ninja/NNPACK-darknet/build/src/ref/convolution-kernel.c.o /home/pi/ninja/NNPACK-darknet/build/src/ref/fully-connected-output.c.o /home/pi/ninja/NNPACK-darknet/build/src/ref/max-pooling-output.c.o /home/pi/ninja/NNPACK-darknet/build/src/ref/softmax-output.c.o /home/pi/ninja/NNPACK-darknet/build/src/ref/relu-output.c.o /home/pi/ninja/NNPACK-darknet/build/src/ref/relu-input-gradient.c.o /home/pi/ninja/NNPACK-darknet/build/test/convolution-output/alexnet.cc.o -lrt /home/pi/ninja/NNPACK-darknet/lib/libnnpack.a /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a /home/pi/ninja/NNPACK-darknet/lib/libgoogletest-core.a /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in functionpthreadpool_create': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:258: undefined reference to pthreadpool_allocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in functionpthreadpool_destroy': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference to pthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference topthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/bin/convolution-output-alexnet-test: internal symbol pthreadpool_deallocate' isn't defined /usr/bin/ld: final link failed: bad value collect2: error: ld returned 1 exit status ninja: build stopped: subcommand failed.

egemenertugrul commented 3 years ago

@mipsan @angmorpri

Thanks everyone, finally it works! Here's the small wiki that I've written.

Great, I wish I had seen this earlier because I spent hours figuring those points out. I ended up writing a guide for building as well, will publish soon. @digitalbrain79 please consider adding @HaroldSP 's guide to README.md.

billhung commented 3 years ago

@HaroldSP I followed you wiki and I still got errors similar to @saifddine pointing the issue to pthreadpool. Any tips on how to solve this?

/home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:258: undefined reference to pthreadpool_allocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/lib/libpthreadpool.a(pthreads.c.o): in function pthreadpool_destroy': /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference to pthreadpool_deallocate' /usr/bin/ld: /home/pi/ninja/NNPACK-darknet/deps/pthreadpool/src/pthreads.c:459: undefined reference to pthreadpool_deallocate'

EDIT: OK, I solved it. Turns out these old instructions 

sudo cp -a lib/* /usr/lib/
sudo cp include/nnpack.h /usr/include/
sudo cp deps/pthreadpool/include/pthreadpool.h /usr/include/

Need to be replaced since the path moved a bit. The one that worked for me was

sudo cp ~/NNPACK/build/deps/pthreadpool/pthreadpool.so /usr/lib/
sudo cp /home/pi/NNPACK/deps/pthreadpool/include/pthreadpool.h /usr/include/
sudo cp /home/pi/NNPACK/include/nnpack.h /usr/include

Otherwise, I followed the new instructions in @HaroldSP 's wiki.

soumyanaveen commented 3 years ago

Same Issue. Getting pthreadpool... error While running the command $NINJA_PATH/ninja. As suggested followed commands mkdir build, cd build, cmake -G Ninja .. (from https://github.com/Maratyszcza/NNPACK ). But getting the error -bash: cmake : command not found. I am using Raspberry Pi 3B. Please help me to resolve this issue.

egemenertugrul commented 3 years ago

Same Issue. Getting pthreadpool... error While running the command $NINJA_PATH/ninja. As suggested followed commands mkdir build, cd build, cmake -G Ninja .. (from https://github.com/Maratyszcza/NNPACK ). But getting the error -bash: cmake : command not found. I am using Raspberry Pi 3B. Please help me to resolve this issue.

https://stackoverflow.com/questions/38658014/ninja-not-found-by-cmake https://github.com/ninja-build/ninja/issues/1222

TLDR; Ninja has to be in the PATH.

I've also gathered a guide here: https://egemenertugrul.github.io/Darknet-NNPACK-on-Raspberry-Pi/ Maybe that helps.