Python: Permission denied using `os.execvp()`

[hastebrot]

I installed Jupyter with pip install jupyter, ignored the compilation error for pyzmq (seems to be an optional dependency) and tried to run Jupyter-Notebook with jupyter notebook.

Error is caused by https://github.com/jupyter/jupyter_core/blob/19d6388f0d/jupyter_core/command.py#L90-L104

[fornwall]

Could you provide exact commands to reproduce the problem?

[hastebrot]

$ apt install python python-dev
$ pip install jupyter
$ jupyter notebook

pip install: Running setup.py install for pyzmq will fail due to missing ifaddrs.h (for #include <ifaddrs.h>). Hmm, I looked into [1] and it seems pyzmq is required.

jupyter notebook: will fail with Permission denied. Arguments for os.execvp() are "jupyter-notebook" and ['notebook'].

[1] https://github.com/jupyter/notebook/blob/master/setup.py

[hastebrot]

jupyter console also doesn't work. Maybe the error with pyzmq revents the commands from being installed.

[hastebrot]

Copied ifaddrs.c and ifaddrs.h from [1] into /data/data/com.termus/files/usr/include/.

pip install pyzmq now fails with: ld: cannot find -lrt.

Hmm, I give up for now.

[1] https://github.com/hastebrot/android-ifaddrs

[fornwall]

I found https://github.com/zeromq/pyzmq/wiki/Cross-compiling-PyZMQ-for-Android about stating PyZMQ >= 13.0 should support cross-compilation to Android without any patches.

Perhaps there has been a regression in the Android support, or otherwise it's not built correctly when building using pip.

[hastebrot]

I haven't tried to cross-compile this as a package for Termux, yet. Since there is this cross-compiling wiki page this might be a better approach.

[timoc]

I've come across the same issue, but pyzmq-15.2.0 fails with the same error. It looks like the ndk-sysroot is missing ifaddrs.h?

[timoc]

Created a gist for a single include file version of the failing android-ifaddrs file https://gist.github.com/timoc/6da1df144cf9dba2049e now compile fails on -lrt

[timoc]

Created a gist for a single include file version of the failing android-ifaddrs file: https://gist.github.com/timoc/6da1df144cf9dba2049e

now compile fails on -lrt which is the posix real time library, which for some reason is not available. Its added to the libraries by the setup.py wrapper, but this can be avoided if libzmq was added to the package list.

[timoc]

So FYI - I tried to compile linzmq locally on my phone, and it had issues other than -lrt.

$ cmake ../libzmq/
-- The C compiler identification is GNU 4.9.3
-- The CXX compiler identification is GNU 4.9.3
-- Check for working C compiler: /data/data/com.termux/files/usr/bin/gcc
-- Check for working C compiler: /data/data/com.termux/files/usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /data/data/com.termux/files/usr/bin/g++
-- Check for working CXX compiler: /data/data/com.termux/files/usr/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test COMPILER_SUPPORTS_CXX11
-- Performing Test COMPILER_SUPPORTS_CXX11 - Success
-- Performing Test COMPILER_SUPPORTS_C11
-- Performing Test COMPILER_SUPPORTS_C11 - Success
-- Using tweetnacl for CURVE security
-- Looking for kqueue
-- Looking for kqueue - not found
-- Looking for epoll_create
-- Looking for epoll_create - found
-- Detected epoll polling method
-- Detected ZMQ Version - 4.2.0
-- Looking for include file ifaddrs.h
-- Looking for include file ifaddrs.h - not found
-- Looking for include file windows.h
-- Looking for include file windows.h - not found
-- Looking for include file sys/uio.h
-- Looking for include file sys/uio.h - found
-- Looking for include file sys/eventfd.h
-- Looking for include file sys/eventfd.h - found
-- Looking for fopen in ws2_32
-- Looking for fopen in ws2_32 - not found
-- Looking for fopen in ws2
-- Looking for fopen in ws2 - not found
-- Looking for fopen in rpcrt4
-- Looking for fopen in rpcrt4 - not found
-- Looking for fopen in iphlpapi
-- Looking for fopen in iphlpapi - not found
-- Looking for SO_PEERCRED
-- Looking for SO_PEERCRED - found
-- Looking for LOCAL_PEERCRED
-- Looking for LOCAL_PEERCRED - not found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE  
-- Looking for clock_gettime
-- Looking for clock_gettime - not found
-- Looking for fork
-- Looking for fork - found
-- Looking for gethrtime
-- Looking for gethrtime - not found
-- Performing Test HAVE_FLAG_-Wall
-- Performing Test HAVE_FLAG_-Wall - Success
-- Performing Test HAVE_FLAG_-Wextra
-- Performing Test HAVE_FLAG_-Wextra - Success
-- Performing Test HAVE_FLAG_-Wno-long-long
-- Performing Test HAVE_FLAG_-Wno-long-long - Success
-- Performing Test HAVE_FLAG_-Wno-uninitialized
-- Performing Test HAVE_FLAG_-Wno-uninitialized - Success
-- Performing Test HAVE_FLAG_-pedantic
-- Performing Test HAVE_FLAG_-pedantic - Success
-- Checking whether SOCK_CLOEXEC is supported
-- Performing Test ZMQ_HAVE_SOCK_CLOEXEC
-- Performing Test ZMQ_HAVE_SOCK_CLOEXEC - Success
-- Checking whether SO_KEEPALIVE is supported
-- Performing Test ZMQ_HAVE_SO_KEEPALIVE
-- Performing Test ZMQ_HAVE_SO_KEEPALIVE - Success
-- Checking whether TCP_KEEPCNT is supported
-- Performing Test ZMQ_HAVE_TCP_KEEPCNT
-- Performing Test ZMQ_HAVE_TCP_KEEPCNT - Success
-- Checking whether TCP_KEEPIDLE is supported
-- Performing Test ZMQ_HAVE_TCP_KEEPIDLE
-- Performing Test ZMQ_HAVE_TCP_KEEPIDLE - Success
-- Checking whether TCP_KEEPINTVL is supported
-- Performing Test ZMQ_HAVE_TCP_KEEPINTVL
-- Performing Test ZMQ_HAVE_TCP_KEEPINTVL - Success
-- Checking whether TCP_KEEPALIVE is supported
-- Performing Test ZMQ_HAVE_TCP_KEEPALIVE
-- Performing Test ZMQ_HAVE_TCP_KEEPALIVE - Failed
-- Found PythonInterp: /data/data/com.termux/files/usr/bin/python (found version "3.5.1") 
-- Could NOT find AsciiDoc (missing:  ASCIIDOC_EXECUTABLE) 
-- Configuring done
-- Generating done
-- Build files have been written to: /data/data/com.termux/files/home/build
$ make
Scanning dependencies of target libzmq
[  0%] Building C object CMakeFiles/libzmq.dir/src/tweetnacl.c.o
[  0%] Building CXX object CMakeFiles/libzmq.dir/src/address.cpp.o
[  0%] Building CXX object CMakeFiles/libzmq.dir/src/client.cpp.o
[  1%] Building CXX object CMakeFiles/libzmq.dir/src/clock.cpp.o
[  1%] Building CXX object CMakeFiles/libzmq.dir/src/ctx.cpp.o
[  1%] Building CXX object CMakeFiles/libzmq.dir/src/curve_client.cpp.o
[  2%] Building CXX object CMakeFiles/libzmq.dir/src/curve_server.cpp.o
[  2%] Building CXX object CMakeFiles/libzmq.dir/src/dealer.cpp.o
[  2%] Building CXX object CMakeFiles/libzmq.dir/src/devpoll.cpp.o
[  2%] Building CXX object CMakeFiles/libzmq.dir/src/dist.cpp.o
[  3%] Building CXX object CMakeFiles/libzmq.dir/src/epoll.cpp.o
[  3%] Building CXX object CMakeFiles/libzmq.dir/src/err.cpp.o
[  3%] Building CXX object CMakeFiles/libzmq.dir/src/fq.cpp.o
[  4%] Building CXX object CMakeFiles/libzmq.dir/src/io_object.cpp.o
[  4%] Building CXX object CMakeFiles/libzmq.dir/src/io_thread.cpp.o
[  4%] Building CXX object CMakeFiles/libzmq.dir/src/ip.cpp.o
[  4%] Building CXX object CMakeFiles/libzmq.dir/src/ipc_address.cpp.o
[  5%] Building CXX object CMakeFiles/libzmq.dir/src/ipc_connecter.cpp.o
[  5%] Building CXX object CMakeFiles/libzmq.dir/src/ipc_listener.cpp.o
[  5%] Building CXX object CMakeFiles/libzmq.dir/src/kqueue.cpp.o
[  6%] Building CXX object CMakeFiles/libzmq.dir/src/lb.cpp.o
[  6%] Building CXX object CMakeFiles/libzmq.dir/src/mailbox.cpp.o
[  6%] Building CXX object CMakeFiles/libzmq.dir/src/mailbox_safe.cpp.o
[  6%] Building CXX object CMakeFiles/libzmq.dir/src/mechanism.cpp.o
[  7%] Building CXX object CMakeFiles/libzmq.dir/src/metadata.cpp.o
[  7%] Building CXX object CMakeFiles/libzmq.dir/src/msg.cpp.o
[  7%] Building CXX object CMakeFiles/libzmq.dir/src/mtrie.cpp.o
[  7%] Building CXX object CMakeFiles/libzmq.dir/src/object.cpp.o
[  8%] Building CXX object CMakeFiles/libzmq.dir/src/options.cpp.o
[  8%] Building CXX object CMakeFiles/libzmq.dir/src/own.cpp.o
[  8%] Building CXX object CMakeFiles/libzmq.dir/src/null_mechanism.cpp.o
[  9%] Building CXX object CMakeFiles/libzmq.dir/src/pair.cpp.o
[  9%] Building CXX object CMakeFiles/libzmq.dir/src/pgm_receiver.cpp.o
[  9%] Building CXX object CMakeFiles/libzmq.dir/src/pgm_sender.cpp.o
[  9%] Building CXX object CMakeFiles/libzmq.dir/src/pgm_socket.cpp.o
[ 10%] Building CXX object CMakeFiles/libzmq.dir/src/pipe.cpp.o
[ 10%] Building CXX object CMakeFiles/libzmq.dir/src/plain_client.cpp.o
[ 10%] Building CXX object CMakeFiles/libzmq.dir/src/plain_server.cpp.o
[ 11%] Building CXX object CMakeFiles/libzmq.dir/src/poll.cpp.o
[ 11%] Building CXX object CMakeFiles/libzmq.dir/src/poller_base.cpp.o
[ 11%] Building CXX object CMakeFiles/libzmq.dir/src/precompiled.cpp.o
[ 11%] Building CXX object CMakeFiles/libzmq.dir/src/proxy.cpp.o
[ 12%] Building CXX object CMakeFiles/libzmq.dir/src/pub.cpp.o
[ 12%] Building CXX object CMakeFiles/libzmq.dir/src/pull.cpp.o
[ 12%] Building CXX object CMakeFiles/libzmq.dir/src/push.cpp.o
[ 13%] Building CXX object CMakeFiles/libzmq.dir/src/random.cpp.o
[ 13%] Building CXX object CMakeFiles/libzmq.dir/src/raw_encoder.cpp.o
[ 13%] Building CXX object CMakeFiles/libzmq.dir/src/raw_decoder.cpp.o
[ 13%] Building CXX object CMakeFiles/libzmq.dir/src/reaper.cpp.o
[ 14%] Building CXX object CMakeFiles/libzmq.dir/src/rep.cpp.o
[ 14%] Building CXX object CMakeFiles/libzmq.dir/src/req.cpp.o
[ 14%] Building CXX object CMakeFiles/libzmq.dir/src/router.cpp.o
[ 15%] Building CXX object CMakeFiles/libzmq.dir/src/select.cpp.o
[ 15%] Building CXX object CMakeFiles/libzmq.dir/src/server.cpp.o
[ 15%] Building CXX object CMakeFiles/libzmq.dir/src/session_base.cpp.o
[ 15%] Building CXX object CMakeFiles/libzmq.dir/src/signaler.cpp.o
[ 16%] Building CXX object CMakeFiles/libzmq.dir/src/socket_base.cpp.o
[ 16%] Building CXX object CMakeFiles/libzmq.dir/src/socks.cpp.o
[ 16%] Building CXX object CMakeFiles/libzmq.dir/src/socks_connecter.cpp.o
[ 16%] Building CXX object CMakeFiles/libzmq.dir/src/stream.cpp.o
[ 17%] Building CXX object CMakeFiles/libzmq.dir/src/stream_engine.cpp.o
[ 17%] Building CXX object CMakeFiles/libzmq.dir/src/sub.cpp.o
[ 17%] Building CXX object CMakeFiles/libzmq.dir/src/tcp.cpp.o
[ 18%] Building CXX object CMakeFiles/libzmq.dir/src/tcp_address.cpp.o
/data/data/com.termux/files/home/libzmq/src/tcp_address.cpp:225:78: warning: unused parameter 'is_src_' [-Wunused-parameter]
 int zmq::tcp_address_t::resolve_nic_name (const char *nic_, bool ipv6_, bool is_src_)
                                                                              ^
[ 18%] Building CXX object CMakeFiles/libzmq.dir/src/tcp_connecter.cpp.o
[ 18%] Building CXX object CMakeFiles/libzmq.dir/src/tcp_listener.cpp.o
[ 18%] Building CXX object CMakeFiles/libzmq.dir/src/thread.cpp.o
/data/data/com.termux/files/home/libzmq/src/thread.cpp:117:49: warning: unused parameter 'priority_' [-Wunused-parameter]
 void zmq::thread_t::setSchedulingParameters(int priority_, int schedulingPolicy_)
                                                 ^
/data/data/com.termux/files/home/libzmq/src/thread.cpp:117:64: warning: unused parameter 'schedulingPolicy_' [-Wunused-parameter]
 void zmq::thread_t::setSchedulingParameters(int priority_, int schedulingPolicy_)
                                                                ^
[ 19%] Building CXX object CMakeFiles/libzmq.dir/src/trie.cpp.o
[ 19%] Building CXX object CMakeFiles/libzmq.dir/src/v1_decoder.cpp.o
[ 19%] Building CXX object CMakeFiles/libzmq.dir/src/v1_encoder.cpp.o
[ 20%] Building CXX object CMakeFiles/libzmq.dir/src/v2_decoder.cpp.o
[ 20%] Building CXX object CMakeFiles/libzmq.dir/src/v2_encoder.cpp.o
[ 20%] Building CXX object CMakeFiles/libzmq.dir/src/xpub.cpp.o
[ 20%] Building CXX object CMakeFiles/libzmq.dir/src/xsub.cpp.o
[ 21%] Building CXX object CMakeFiles/libzmq.dir/src/zmq.cpp.o
[ 21%] Building CXX object CMakeFiles/libzmq.dir/src/zmq_utils.cpp.o
[ 21%] Building CXX object CMakeFiles/libzmq.dir/src/decoder_allocators.cpp.o
[ 22%] Building CXX object CMakeFiles/libzmq.dir/src/socket_poller.cpp.o
[ 22%] Building CXX object CMakeFiles/libzmq.dir/src/timers.cpp.o
[ 22%] Building CXX object CMakeFiles/libzmq.dir/src/radio.cpp.o
[ 22%] Building CXX object CMakeFiles/libzmq.dir/src/dish.cpp.o
[ 23%] Building CXX object CMakeFiles/libzmq.dir/src/udp_engine.cpp.o
[ 23%] Building CXX object CMakeFiles/libzmq.dir/src/udp_address.cpp.o
[ 23%] Linking CXX shared library lib/libzmq.so
[ 23%] Built target libzmq
Scanning dependencies of target inproc_lat
[ 23%] Building CXX object CMakeFiles/inproc_lat.dir/perf/inproc_lat.cpp.o
[ 23%] Linking CXX executable bin/inproc_lat
lib/libzmq.so.4.2.0: undefined reference to `std::__throw_length_error(char const*)'
lib/libzmq.so.4.2.0: undefined reference to `std::__throw_logic_error(char const*)'
lib/libzmq.so.4.2.0: undefined reference to `vtable for std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >'
lib/libzmq.so.4.2.0: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&)'
lib/libzmq.so.4.2.0: undefined reference to `std::_Rb_tree_increment(std::_Rb_tree_node_base const*)'
lib/libzmq.so.4.2.0: undefined reference to `vtable for std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >'
lib/libzmq.so.4.2.0: undefined reference to `std::string::_M_mutate(unsigned int, unsigned int, unsigned int)'
lib/libzmq.so.4.2.0: undefined reference to `std::__throw_bad_alloc()'
lib/libzmq.so.4.2.0: undefined reference to `vtable for std::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >'
lib/libzmq.so.4.2.0: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::string const&, unsigned int, unsigned int)'
lib/libzmq.so.4.2.0: undefined reference to `vtable for std::basic_streambuf<char, std::char_traits<char> >'
lib/libzmq.so.4.2.0: undefined reference to `vtable for std::basic_ios<char, std::char_traits<char> >'
lib/libzmq.so.4.2.0: undefined reference to `std::ios_base::~ios_base()'
lib/libzmq.so.4.2.0: undefined reference to `std::string::rfind(char const*, unsigned int, unsigned int) const'
lib/libzmq.so.4.2.0: undefined reference to `std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, int)'
lib/libzmq.so.4.2.0: undefined reference to `std::_Rb_tree_increment(std::_Rb_tree_node_base*)'
lib/libzmq.so.4.2.0: undefined reference to `std::string::swap(std::string&)'
lib/libzmq.so.4.2.0: undefined reference to `std::string::find(char const*, unsigned int, unsigned int) const'
lib/libzmq.so.4.2.0: undefined reference to `std::_Rb_tree_decrement(std::_Rb_tree_node_base*)'
lib/libzmq.so.4.2.0: undefined reference to `std::string::assign(std::string const&)'
lib/libzmq.so.4.2.0: undefined reference to `std::ios_base::ios_base()'
lib/libzmq.so.4.2.0: undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
lib/libzmq.so.4.2.0: undefined reference to `std::string::_Rep::_S_create(unsigned int, unsigned int, std::allocator<char> const&)'
lib/libzmq.so.4.2.0: undefined reference to `std::string::_Rep::_S_empty_rep_storage'
lib/libzmq.so.4.2.0: undefined reference to `VTT for std::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >'
lib/libzmq.so.4.2.0: undefined reference to `std::string::append(std::string const&)'
lib/libzmq.so.4.2.0: undefined reference to `std::locale::~locale()'
lib/libzmq.so.4.2.0: undefined reference to `std::basic_ios<char, std::char_traits<char> >::init(std::basic_streambuf<char, std::char_traits<char> >*)'
lib/libzmq.so.4.2.0: undefined reference to `std::string::assign(char const*, unsigned int)'
lib/libzmq.so.4.2.0: undefined reference to `std::string::rfind(char, unsigned int) const'
lib/libzmq.so.4.2.0: undefined reference to `std::locale::locale()'
lib/libzmq.so.4.2.0: undefined reference to `std::_Rb_tree_rebalance_for_erase(std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)'
lib/libzmq.so.4.2.0: undefined reference to `std::string::reserve(unsigned int)'
lib/libzmq.so.4.2.0: undefined reference to `std::string::_M_leak_hard()'
lib/libzmq.so.4.2.0: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned int, std::allocator<char> const&)'
lib/libzmq.so.4.2.0: undefined reference to `std::string::compare(char const*) const'
lib/libzmq.so.4.2.0: undefined reference to `VTT for std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >'
lib/libzmq.so.4.2.0: undefined reference to `std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)'
lib/libzmq.so.4.2.0: undefined reference to `std::ostream& std::ostream::_M_insert<unsigned long>(unsigned long)'
lib/libzmq.so.4.2.0: undefined reference to `std::string::_Rep::_M_destroy(std::allocator<char> const&)'
lib/libzmq.so.4.2.0: undefined reference to `std::string::append(char const*, unsigned int)'
lib/libzmq.so.4.2.0: undefined reference to `std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&)'
lib/libzmq.so.4.2.0: undefined reference to `std::ostream::operator<<(int)'
collect2: error: ld returned 1 exit status
CMakeFiles/inproc_lat.dir/build.make:95: recipe for target 'bin/inproc_lat' failed
make[2]: *** [bin/inproc_lat] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/inproc_lat.dir/all' failed
make[1]: *** [CMakeFiles/inproc_lat.dir/all] Error 2
Makefile:138: recipe for target 'all' failed
make: *** [all] Error 2
$ 

[garwiz]

I've got this working with local libzmq installation which gets used by pyzmq. Dev version is used due to broken cmake build in 4.1.5 tarball. See https://github.com/zeromq/libzmq/issues/2096

apt install python python-dev clang cmake make git
git clone https://github.com/zeromq/zeromq4-1.git
mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX:PATH=/data/data/com.termux/files/usr ../zeromq4-1
make install
pip install jupyter-console ipykernel

ipython3 kernel
# and in other session 
jupyter-console --existing

Version 4.1.5 builds with autoconf handwork on android and runs fine as well. I will add a libzmq and libzmq-dev package for termux if you agree.

[timoc]

+1 for me at least.

[hastebrot]

Also note, there are PRs for a ØMQ (#270) and PyZMQ package (#271).

[garwiz]

Regarding PR https://github.com/termux/termux-packages/pull/271 and a dedicated pyzmq package. I don't think termux should start providing python-packages which are available from pypi. With PR https://github.com/termux/termux-packages/issues/437 you only need to do apt install clang python-dev libzmq-dev to get pip install pyzmq working.

[fornwall]

I don't think termux should start providing python-packages which are available from pypi.

@garwiz Agreed! PR https://github.com/termux/termux-packages/pull/437 is now merged, so I'm closing this issue. Thanks!