nbansal90 commented 5 months ago

Hello,

I am trying to use Executorch for my custom model. I am able to perform all the steps mentioned at https://pytorch.org/executorch/stable/getting-started-setup.html. But when I run my program the program is always stuck at Input Prepared step for ever.

(executorch) n.bansal1@SRA0401:~/projects/executorch$ ./cmake-out/executor_runner --model_path ../utils/model.pte I 00:00:00.073575 executorch:executor_runner.cpp:73] Model file ../utils/model.pte is loaded. I 00:00:00.073611 executorch:executor_runner.cpp:82] Using method forward I 00:00:00.073616 executorch:executor_runner.cpp:129] Setting up planned buffer 0, size 1161822208. I 00:00:00.896979 executorch:executor_runner.cpp:152] Method loaded. I 00:00:00.897851 executorch:executor_runner.cpp:162] Inputs prepared.

Seeing the executor_runner.cpp file:

auto inputs = util::prepare_input_tensors(*method);
  ET_CHECK_MSG(
      inputs.ok(),
      "Could not prepare inputs: 0x%" PRIx32,
      (uint32_t)inputs.error());
  ET_LOG(Info, "Inputs prepared.");

  // Run the model.
  Error status = method->execute();  #<---------------------------------
  ET_CHECK_MSG(
      status == Error::Ok,
      "Execution of method %s failed with status 0x%" PRIx32,
      method_name,
      (uint32_t)status);
  ET_LOG(Info, "Model executed successfully.");

It is supposed to be executing the model using method->execute but somehow the program is stuck for ever at this step. Please suggest what could be probable issue.

Regards,

digantdesai commented 5 months ago

I am assuming the toy/example models in the documentation works ok?

Can you try this under a C++ debugger and see where is it getting stuck, you model is rather large ~1 GIB, so any other signals from the system which may indicate what's going on?

nbansal90 commented 5 months ago

Hey @digantdesai, Thanks for the prompt reply!

I am currently just executing this under Linux terminal and not using any IDE. What exactly do you mean when you say try under a C++ debugger ?

Regards, Nitin Bansal

cccclai commented 5 months ago

Did you lower the model to any backend, like XNNPACK backend? This is our default backend for CPU case. If it's not lowered, it defaults to the portable kernel, which aims for portable across all hardware but not performant.

nbansal90 commented 5 months ago

@cccclai @digantdesai The model did run eventually but it took more than an hour to finish running. @cccclai As I tried building for an XNNPACK backend. I find that after cmake --build cmake-out -j9 --target install --config Release step. i find xnn_executor_runner file is missing at ./cmake-out/backends/xnnpack/ folder location.

Steps Followed:

Got a .pte model with XnnPackBackend support. (edge = edge.to_backend(XnnpackPartitioner()))
Configured make without any error for XnnPack backend
Build finishes 100% but with 2 errors: (cmake --build cmake-out -j9 --target install --config Release)

`[100%] Building C object backends/xnnpack/third-party/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/exp2minus-k-over-8.c.o [100%] Building C object backends/xnnpack/third-party/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/exp2minus-k-over-16.c.o [100%] Building C object backends/xnnpack/third-party/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/exp2minus-k-over-32.c.o [100%] Building C object backends/xnnpack/third-party/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/exp2minus-k-over-64.c.o [100%] Building C object backends/xnnpack/third-party/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/exp2minus-k-over-2048.c.o [100%] Building C object backends/xnnpack/third-party/XNNPACK/CMakeFiles/microkernels-all.dir/src/tables/vlog.c.o [100%] Built target microkernels-all gmake: *** [Makefile:136: all] Error

Content of CMakeError.txt Performing C SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output: Change Dir: /home/n.bansal1/projects/executorch/cmake-out/CMakeFiles/CMakeTmp Run Build Command(s):/usr/bin/gmake -f Makefile cmTC_d1626/fast && /usr/bin/gmake -f CMakeFiles/cmTC_d1626.dir/build.make CMakeFiles/cmTC_d1626.dir/build gmake[1]: Entering directory '/home/n.bansal1/projects/executorch/cmake-out/CMakeFiles/CMakeTmp' Building C object CMakeFiles/cmTC_d1626.dir/src.c.o /home/n.bansal1/anaconda3/envs/pytorch_vulkan/bin/cc -DCMAKE_HAVE_LIBC_PTHREAD -o CMakeFiles/cmTC_d1626.dir/src.c.o -c /home/n.bansal1/projects/executorch/cmake-out/CMakeFiles/CMakeTmp/src.c Linking C executable cmTC_d1626 /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_d1626.dir/link.txt --verbose=1 /home/n.bansal1/anaconda3/envs/pytorch_vulkan/bin/cc CMakeFiles/cmTC_d1626.dir/src.c.o -o cmTC_d1626 /home/n.bansal1/anaconda3/envs/pytorch_vulkan/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: CMakeFiles/cmTC_d1626.dir/src.c.o: in function main': src.c:(.text+0x2f): undefined reference topthread_create' /home/n.bansal1/anaconda3/envs/pytorch_vulkan/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: src.c:(.text+0x3b): undefined reference to pthread_detach' /home/n.bansal1/anaconda3/envs/pytorch_vulkan/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: src.c:(.text+0x47): undefined reference topthread_cancel' /home/n.bansal1/anaconda3/envs/pytorch_vulkan/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: src.c:(.text+0x58): undefined reference to pthread_join' /home/n.bansal1/anaconda3/envs/pytorch_vulkan/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: src.c:(.text+0x6c): undefined reference topthread_atfork' collect2: error: ld returned 1 exit status gmake[1]: [CMakeFiles/cmTC_d1626.dir/build.make:99: cmTC_d1626] Error 1 gmake[1]: Leaving directory '/home/n.bansal1/projects/executorch/cmake-out/CMakeFiles/CMakeTmp' gmake: [Makefile:127: cmTC_d1626/fast] Error 2

digantdesai commented 5 months ago

somehow you are missing -pthread not sure. @mcr229 is the document link up to date?

nbansal90 commented 5 months ago

@digantdesai @cccclai Just a side question. Am I supposed to be running this command: ./install_requirements.sh --pybind <coreml | mps | xnnpack> while setting up the repository for running on XnnPack backend or the configuration mentioned at https://pytorch.org/executorch/stable/tutorial-xnnpack-delegate-lowering.html are sufficient.

nbansal90 commented 4 months ago

Added Information:

I have given this try multiple times based on some suggestions over the internet. But does not seem to workout. I discovered that while doing the cmake configuration(cmake -DCMAKE_PREFIX_PATH=/home/n.bansal1/anaconda3/envs/executorch -DCMAKE_INSTALL_PREFIX=cmake-out -DCMAKE_BUILD_TYPE=Release -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON -DEXECUTORCH_BUILD_XNNPACK=ON -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON -DEXECUTORCH_ENABLE_LOGGING=ON -DPYTHON_EXECUTABLE=python -Bcmake-out .) itself, the CMakeError.log is generated, and currently I am seeing this as the error:

Determining if the strtoull_l exist failed with the following output: Change Dir: /home/n.bansal1/projects/executorch/cmake-out/CMakeFiles/CMakeScratch/TryCompile-pKQCsj

Run Build Command(s):/usr/bin/gmake -f Makefile cmTC_e24a0/fast && /usr/bin/gmake -f CMakeFiles/cmTC_e24a0.dir/build.make CMakeFiles/cmTC_e24a0.dir/build gmake[1]: Entering directory '/home/n.bansal1/projects/executorch/cmake-out/CMakeFiles/CMakeScratch/TryCompile-pKQCsj' Building CXX object CMakeFiles/cmTC_e24a0.dir/CheckSymbolExists.cxx.o /usr/bin/c++ -o CMakeFiles/cmTC_e24a0.dir/CheckSymbolExists.cxx.o -c /home/n.bansal1/projects/executorch/cmake-out/CMakeFiles/CMakeScratch/TryCompile-pKQCsj/CheckSymbolExists.cxx Linking CXX executable cmTC_e24a0 /home/n.bansal1/.local/lib/python3.10/site-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmTC_e24a0.dir/link.txt --verbose=1 /usr/bin/c++ CMakeFiles/cmTC_e24a0.dir/CheckSymbolExists.cxx.o -o cmTC_e24a0 /usr/bin/ld: CMakeFiles/cmTC_e24a0.dir/CheckSymbolExists.cxx.o: in function main': CheckSymbolExists.cxx:(.text+0x1f): undefined reference to__isoc23_strtoull_l' collect2: error: ld returned 1 exit status gmake[1]: [CMakeFiles/cmTC_e24a0.dir/build.make:99: cmTC_e24a0] Error 1 gmake[1]: Leaving directory '/home/n.bansal1/projects/executorch/cmake-out/CMakeFiles/CMakeScratch/TryCompile-pKQCsj' gmake: [Makefile:127: cmTC_e24a0/fast] Error 2

File CheckSymbolExists.cxx: / /

include

int main(int argc, char** argv) { (void)argv;

ifndef strtoull_l

return ((int*)(&strtoull_l))[argc];

else

(void)argc; return 0;

endif

} Determining if the strtoll exist failed with the following output: Change Dir: /home/n.bansal1/projects/executorch/cmake-out/CMakeFiles/CMakeScratch/TryCompile-OeJtkG`

During the execution of cmake configuration screen logs I see these 'warning' message:

Generating operator lib: -- LIB_NAME: portable_ops_lib -- KERNEL_LIBS: portable_kernels -- DEPS: executorch -- Looking for C++ include unistd.h -- Looking for C++ include unistd.h - found -- Looking for C++ include stdint.h -- Looking for C++ include stdint.h - found -- Looking for C++ include inttypes.h -- Looking for C++ include inttypes.h - found -- Looking for C++ include sys/types.h -- Looking for C++ include sys/types.h - found -- Looking for C++ include sys/stat.h -- Looking for C++ include sys/stat.h - found -- Looking for C++ include fnmatch.h -- Looking for C++ include fnmatch.h - found -- Looking for C++ include stddef.h -- Looking for C++ include stddef.h - found -- Check size of uint32_t -- Check size of uint32_t - done -- Looking for strtoll -- Looking for strtoll - not found -- Looking for strtoq -- Looking for strtoq - not found

nbansal90 commented 4 months ago

Hey @digantdesai, Please let me know if there is extra information is required. I am still stuck at this juncture.

digantdesai commented 4 months ago

__isoc23_strtoull_l should be provided by the GLIBC on your system. Have you tried building with EXECUTORCH_BUILD_XNNPACK=OFF? Does it work?

nbansal90 commented 4 months ago

Closing this issue for now. As seems like the current error is due to something wrong on my system.

pytorch / executorch

xnn_executer_runner missing after building for XnnPackDelegate #3919

include

ifndef strtoull_l

else

endif