ContinuumIO / anaconda-issues

Anaconda issue tracking
646 stars 220 forks source link

Interaction between suid-bit and weakref.py (Linux Mint/Ubuntu) - Anaconda python2.7 #9200

Open mazerj opened 6 years ago

mazerj commented 6 years ago

Actual Behavior

Anaconda distribution binary behaves differently when importing weakref.py (triggered by import numpy as np, for example) when the suid-bit is set.

This is really obscure, but I'm running the python interpreter from a thin C wrapper program and if the wrapper is suid-root, import weakref.py fails. Same program, with suid-bit cleared and if works. Doesn't happen with the stock python2.7 interpreter provided by distribution. Interestingly, if the suid-bit is set, but seteuid() is used to change the UID back to the a non-root value, it still fails. Demo below:

mazer@glacier $ l min mins -rwxr-xr-x 1 mazer mlab 8816 Apr 18 14:08 min ---s--x--x 1 root mlab 8816 Apr 18 14:08 mins [2] mazer@glacier $ ./min # suid bit is NOT set, execution is normal: not suid root. <module 'numpy' from '/auto/home/mazer/anaconda2/lib/python2.7/site-packages/numpy/init.pyc'> [2] mazer@glacier $ ./mins # suid bit is set, import of weakpref files Traceback (most recent call last): File "", line 1, in File "/auto/home/mazer/anaconda2/lib/python2.7/site-packages/numpy/init.py", line 142, in from . import add_newdocs File "/auto/home/mazer/anaconda2/lib/python2.7/site-packages/numpy/add_newdocs.py", line 13, in from numpy.lib import add_newdoc File "/auto/home/mazer/anaconda2/lib/python2.7/site-packages/numpy/lib/init.py", line 8, in from .type_check import * File "/auto/home/mazer/anaconda2/lib/python2.7/site-packages/numpy/lib/type_check.py", line 11, in import numpy.core.numeric as _nx File "/auto/home/mazer/anaconda2/lib/python2.7/site-packages/numpy/core/init.py", line 74, in from numpy.testing import _numpy_tester File "/auto/home/mazer/anaconda2/lib/python2.7/site-packages/numpy/testing/init.py", line 10, in from unittest import TestCase File "/auto/home/mazer/anaconda2/lib/python2.7/unittest/init.py", line 64, in from .main import TestProgram, main File "/auto/home/mazer/anaconda2/lib/python2.7/unittest/main.py", line 7, in from . import loader, runner File "/auto/home/mazer/anaconda2/lib/python2.7/unittest/runner.py", line 7, in from .signals import registerResult File "/auto/home/mazer/anaconda2/lib/python2.7/unittest/signals.py", line 2, in import weakref File "/auto/home/mazer/anaconda2/lib/python2.7/weakref.py", line 14, in from _weakref import ( ImportError: cannot import name _remove_dead_weakref

Expected Behavior

Execution should be same regardless of suid-bit status.

Steps to Reproduce

Compile and run the following program (makefile below):

// MAKE SURE TO SET newav[0] TO LOCATION OF YOUR PYTHON EXE
#include <stdio.h>
#include <unistd.h>

int main()
{
  char *newav[4];
  int newac = 3;

  newav[0] = "/auto/home/mazer/anaconda2/bin/python";
  newav[1] = "-c";
  newav[2] = "import numpy as np; print np";
  newav[3] = NULL;      /* no necessary! */

  if (geteuid() != 0) {
    fprintf(stderr, "not suid root.\n");
  }
  execvp(newav[0], newav);
}
all:
    $(CC) min.c -o min
    cp min mins
    sudo chown root mins
    sudo chmod 4111 mins
Anaconda or Miniconda version:

Anaconda2-5.1.0-Linux-x86_64

Operating System:
conda info
environment : None
       user config file : /auto/home/mazer/.condarc
 populated config files : /auto/home/mazer/.condarc
          conda version : 4.4.10
    conda-build version : 3.4.1
         python version : 2.7.14.final.0
       base environment : /auto/home/mazer/anaconda2  (writable)
           channel URLs : https://conda.binstar.org/erik/linux-64
                          https://conda.binstar.org/erik/noarch
                          https://repo.continuum.io/pkgs/main/linux-64
                          https://repo.continuum.io/pkgs/main/noarch
                          https://repo.continuum.io/pkgs/free/linux-64
                          https://repo.continuum.io/pkgs/free/noarch
                          https://repo.continuum.io/pkgs/r/linux-64
                          https://repo.continuum.io/pkgs/r/noarch
                          https://repo.continuum.io/pkgs/pro/linux-64
                          https://repo.continuum.io/pkgs/pro/noarch
          package cache : /auto/home/mazer/anaconda2/pkgs
                          /auto/home/mazer/.conda/pkgs
       envs directories : /auto/home/mazer/anaconda2/envs
                          /auto/home/mazer/.conda/envs
               platform : linux-64
             user-agent : conda/4.4.10 requests/2.18.4 CPython/2.7.14 Linux/4.10.0-33-generic linuxmint/18.2 glibc/2.23
                UID:GID : 583:705
             netrc file : None
           offline mode : False
conda list --show-channel-urls
# packages in environment at /auto/home/mazer/anaconda2:
#
# Name                    Version                   Build  Channel
_ipyw_jlab_nb_ext_conf    0.1.0            py27h08a7f0c_0    defaults
alabaster                 0.7.10           py27he5a193a_0    defaults
anaconda                  5.1.0                    py27_2    defaults
anaconda-client           1.6.9                    py27_0    defaults
anaconda-navigator        1.7.0                    py27_0    defaults
anaconda-project          0.8.2            py27h236b58a_0    defaults
asn1crypto                0.24.0                   py27_0    defaults
astroid                   1.6.1                    py27_0    defaults
astropy                   2.0.3            py27h14c3975_0    defaults
attrs                     17.4.0                   py27_0    defaults
babel                     2.5.3                    py27_0    defaults
backports                 1.0              py27h63c9359_1    defaults
backports.functools_lru_cache 1.4              py27he8db605_1    defaults
backports.shutil_get_terminal_size 1.0.0            py27h5bc021e_2    defaults
backports_abc             0.5              py27h7b3c97b_0    defaults
beautifulsoup4            4.6.0            py27h3f86ba9_1    defaults
bitarray                  0.8.1            py27h14c3975_1    defaults
bkcharts                  0.2              py27h241ae91_0    defaults
blaze                     0.11.3           py27h5f341da_0    defaults
bleach                    2.1.2                    py27_0    defaults
bokeh                     0.12.13          py27h5233db4_0    defaults
boto                      2.48.0           py27h9556ac2_1    defaults
bottleneck                1.2.1            py27h21b16a3_0    defaults
bzip2                     1.0.6                h9a117a8_4    defaults
ca-certificates           2017.08.26           h1d4fec5_0    defaults
cairo                     1.14.12              h77bcde2_0    defaults
cdecimal                  2.3              py27h14c3975_3    defaults
certifi                   2018.1.18                py27_0    defaults
cffi                      1.11.4           py27h9745a5d_0    defaults
chardet                   3.0.4            py27hfa10054_1    defaults
click                     6.7              py27h4225b90_0    defaults
cloudpickle               0.5.2                    py27_1    defaults
clyent                    1.2.2            py27h7276e6c_1    defaults
colorama                  0.3.9            py27h5cde069_0    defaults
conda                     4.4.10                   py27_0    defaults
conda-build               3.4.1                    py27_0    defaults
conda-env                 2.6.0                h36134e3_1    defaults
conda-verify              2.0.0            py27hf052a9d_0    defaults
configparser              3.5.0            py27h5117587_0    defaults
contextlib2               0.5.5            py27hbf4c468_0    defaults
cryptography              2.1.4            py27h6697b16_0    defaults
curl                      7.58.0               h84994c4_0    defaults
cycler                    0.10.0           py27hc7354d3_0    defaults
cython                    0.27.3           py27hc56b35e_0    defaults
cytoolz                   0.9.0            py27h14c3975_0    defaults
dask                      0.16.1                   py27_0    defaults
dask-core                 0.16.1                   py27_0    defaults
datashape                 0.5.4            py27hf507385_0    defaults
dbus                      1.12.2               hc3f9b76_1    defaults
decorator                 4.2.1                    py27_0    defaults
distributed               1.20.2                   py27_0    defaults
docutils                  0.14             py27hae222c1_0    defaults
entrypoints               0.2.3            py27h502b47d_2    defaults
enum34                    1.1.6            py27h99a27e9_1    defaults
et_xmlfile                1.0.1            py27h75840f5_0    defaults
expat                     2.2.5                he0dffb1_0    defaults
fastcache                 1.0.2            py27h14c3975_2    defaults
filelock                  2.0.13           py27h61a9c69_0    defaults
flask                     0.12.2           py27h6d5c1cd_0    defaults
flask-cors                3.0.3            py27h1a8a27f_0    defaults
fontconfig                2.12.4               h88586e7_1    defaults
freetype                  2.8                  hab7d2ae_1    defaults
funcsigs                  1.0.2            py27h83f16ab_0    defaults
functools32               3.2.3.2          py27h4ead58f_1    defaults
futures                   3.2.0            py27h7b459c0_0    defaults
get_terminal_size         1.0.0                haa9412d_0    defaults
gevent                    1.2.2            py27h475ea6a_0    defaults
glib                      2.53.6               h5d9569c_2    defaults
glob2                     0.6              py27hcea9cbd_0    defaults
gmp                       6.1.2                h6c8ec71_1    defaults
gmpy2                     2.0.8            py27h4cf3fa8_2    defaults
graphite2                 1.3.10               hf63cedd_1    defaults
greenlet                  0.4.12           py27hac09c53_0    defaults
grin                      1.2.1                    py27_4    defaults
gst-plugins-base          1.12.4               h33fb286_0    defaults
gstreamer                 1.12.4               hb53b477_0    defaults
h5py                      2.7.1            py27h2697762_0    defaults
harfbuzz                  1.7.4                hc5b324e_0    defaults
hdf5                      1.10.1               h9caa474_1    defaults
heapdict                  1.0.0                    py27_2    defaults
html5lib                  1.0.1            py27h5233db4_0    defaults
icu                       58.2                 h9c2bf20_1    defaults
idna                      2.6              py27h5722d68_1    defaults
imageio                   2.2.0            py27hf108a7f_0    defaults
imagesize                 0.7.1            py27hd17bf80_0    defaults
intel-openmp              2018.0.0             hc7b2577_8    defaults
ipaddress                 1.0.19                   py27_0    defaults
ipykernel                 4.8.0                    py27_0    defaults
ipython                   5.4.1                    py27_2    defaults
ipython_genutils          0.2.0            py27h89fb69b_0    defaults
ipywidgets                7.1.1                    py27_0    defaults
isort                     4.2.15           py27hcfa4749_0    defaults
itsdangerous              0.24             py27hb8295c1_1    defaults
jbig                      2.1                  hdba287a_0    defaults
jdcal                     1.3              py27h2cc5433_0    defaults
jedi                      0.11.1                   py27_0    defaults
jinja2                    2.10             py27h4114e70_0    defaults
jpeg                      9b                   h024ee3a_2    defaults
jsonschema                2.6.0            py27h7ed5aa4_0    defaults
jupyter                   1.0.0                    py27_4    defaults
jupyter_client            5.2.2                    py27_0    defaults
jupyter_console           5.2.0            py27hc6bee7e_1    defaults
jupyter_core              4.4.0            py27h345911c_0    defaults
jupyterlab                0.31.5                   py27_0    defaults
jupyterlab_launcher       0.10.2                   py27_0    defaults
lazy-object-proxy         1.3.1            py27h682c727_0    defaults
libcurl                   7.58.0               h1ad7b7a_0    defaults
libedit                   3.1                  heed3624_0    defaults
libffi                    3.2.1                hd88cf55_4    defaults
libgcc-ng                 7.2.0                h7cc24e2_2    defaults
libgfortran-ng            7.2.0                h9f7466a_2    defaults
libpng                    1.6.34               hb9fc6fc_0    defaults
libsodium                 1.0.15               hf101ebd_0    defaults
libssh2                   1.8.0                h9cfc8f7_4    defaults
libstdcxx-ng              7.2.0                h7a57d05_2    defaults
libtiff                   4.0.9                h28f6b97_0    defaults
libtool                   2.4.6                h544aabb_3    defaults
libxcb                    1.12                 hcd93eb1_4    defaults
libxml2                   2.9.7                h26e45fe_0    defaults
libxslt                   1.1.32               h1312cb7_0    defaults
llvmlite                  0.21.0           py27h6dad3f9_0    defaults
locket                    0.2.0            py27h73929a2_1    defaults
lxml                      4.1.1            py27hf71bdeb_1    defaults
lzo                       2.10                 h49e0be7_2    defaults
markupsafe                1.0              py27h97b2822_1    defaults
matplotlib                2.1.2            py27h0e671d2_0    defaults
mccabe                    0.6.1            py27h0e7c7be_1    defaults
mistune                   0.8.3                    py27_0    defaults
mkl                       2018.0.1             h19d6760_4    defaults
mkl-service               1.1.2            py27hb2d42c5_4    defaults
mpc                       1.0.3                hec55b23_5    defaults
mpfr                      3.1.5                h11a74b3_2    defaults
mpmath                    1.0.0            py27h9669132_2    defaults
msgpack-python            0.5.1            py27h6bb024c_0    defaults
multipledispatch          0.4.9            py27h9b5f95a_0    defaults
navigator-updater         0.1.0            py27h0f9cd39_0    defaults
nbconvert                 5.3.1            py27he041f76_0    defaults
nbformat                  4.4.0            py27hed7f2b2_0    defaults
ncurses                   6.0                  h9df7e31_2    defaults
networkx                  2.1                      py27_0    defaults
nltk                      3.2.5            py27hec5f4de_0    defaults
nose                      1.3.7            py27heec2199_2    defaults
notebook                  5.4.0                    py27_0    defaults
numba                     0.36.2          np114py27he2868da_0    defaults
numexpr                   2.6.4            py27hd318778_0    defaults
numpy                     1.14.0           py27h3dfced4_1    defaults
numpydoc                  0.7.0            py27h9647a75_0    defaults
odo                       0.5.1            py27h9170de3_0    defaults
olefile                   0.45.1                   py27_0    defaults
openpyxl                  2.4.10                   py27_0    defaults
openssl                   1.0.2n               hb7f436b_0    defaults
packaging                 16.8             py27h5e07c7c_1    defaults
pandas                    0.22.0           py27hf484d3e_0    defaults
pandoc                    1.19.2.1             hea2e7c5_1    defaults
pandocfilters             1.4.2            py27h428e1e5_1    defaults
pango                     1.41.0               hd475d92_0    defaults
parso                     0.1.1            py27h718acc2_0    defaults
partd                     0.3.8            py27h4e55004_0    defaults
patchelf                  0.9                  hf79760b_2    defaults
path.py                   10.5             py27hefe4bee_0    defaults
pathlib2                  2.3.0            py27h6e9d198_0    defaults
patsy                     0.5.0                    py27_0    defaults
pcre                      8.41                 hc27e229_1    defaults
pep8                      1.7.1                    py27_0    defaults
pexpect                   4.3.1                    py27_0    defaults
pickleshare               0.7.4            py27h09770e1_0    defaults
pillow                    5.0.0            py27h3deb7b8_0    defaults
pip                       9.0.1            py27ha730c48_4    defaults
pixman                    0.34.0               hceecf20_3    defaults
pkginfo                   1.4.1            py27hee1a9ad_1    defaults
pluggy                    0.6.0            py27h1f4f128_0    defaults
ply                       3.10             py27hd6d9ae5_0    defaults
prompt_toolkit            1.0.15           py27h1b593e1_0    defaults
psutil                    5.4.3            py27h14c3975_0    defaults
ptyprocess                0.5.2            py27h4ccb14c_0    defaults
py                        1.5.2            py27h203d672_0    defaults
pycairo                   1.15.4           py27h1b9232e_1    defaults
pycodestyle               2.3.1            py27h904819d_0    defaults
pycosat                   0.6.3            py27ha4109ae_0    defaults
pycparser                 2.18             py27hefa08c5_1    defaults
pycrypto                  2.6.1            py27h14c3975_7    defaults
pycurl                    7.43.0.1         py27hb7f436b_0    defaults
pyflakes                  1.6.0            py27h904a57d_0    defaults
pygame                    1.9.3                     <pip>
pygments                  2.2.0            py27h4a8b6f5_0    defaults
pylint                    1.8.2                    py27_0    defaults
pyodbc                    4.0.22           py27hf484d3e_0    defaults
PyOpenGL                  3.1.0                     <pip>
pyopenssl                 17.5.0           py27hcee3be0_0    defaults
pyparsing                 2.2.0            py27hf1513f8_1    defaults
pyqt                      5.6.0            py27h4b1e83c_5    defaults
pysocks                   1.6.7            py27he2db6d2_1    defaults
pytables                  3.4.2            py27h1f7bffc_2    defaults
pytest                    3.3.2                    py27_0    defaults
python                    2.7.14              h1571d57_29    defaults
python-dateutil           2.6.1            py27h4ca5741_1    defaults
pytz                      2017.3           py27h001bace_0    defaults
pywavelets                0.5.2            py27hecda097_0    defaults
pyyaml                    3.12             py27h2d70dd7_1    defaults
pyzmq                     16.0.3           py27hc579512_0    defaults
qt                        5.6.2               h974d657_12    defaults
qtawesome                 0.4.4            py27hd7914c3_0    defaults
qtconsole                 4.3.1            py27hc444b0d_0    defaults
qtpy                      1.3.1            py27h63d3751_0    defaults
readline                  7.0                  ha6073c6_4    defaults
requests                  2.18.4           py27hc5b0589_1    defaults
rope                      0.10.7           py27hfe459b0_0    defaults
ruamel_yaml               0.15.35          py27h14c3975_1    defaults
scandir                   1.6              py27hf7388dc_0    defaults
scikit-image              0.13.1           py27h14c3975_1    defaults
scikit-learn              0.19.1           py27h445a80a_0    defaults
scipy                     1.0.0            py27hf5f0f52_0    defaults
seaborn                   0.8.1            py27h633ea1e_0    defaults
send2trash                1.4.2                    py27_0    defaults
setuptools                38.4.0                   py27_0    defaults
simplegeneric             0.8.1                    py27_2    defaults
singledispatch            3.4.0.3          py27h9bcb476_0    defaults
sip                       4.18.1           py27he9ba0ab_2    defaults
six                       1.11.0           py27h5f960f1_1    defaults
snowballstemmer           1.2.1            py27h44e2768_0    defaults
sortedcollections         0.5.3            py27h135218e_0    defaults
sortedcontainers          1.5.9                    py27_0    defaults
sphinx                    1.6.6                    py27_0    defaults
sphinxcontrib             1.0              py27h1512b58_1    defaults
sphinxcontrib-websupport  1.0.1            py27hf906f22_1    defaults
spyder                    3.2.6                    py27_0    defaults
sqlalchemy                1.2.1            py27h14c3975_0    defaults
sqlite                    3.22.0               h1bed415_0    defaults
ssl_match_hostname        3.5.0.1          py27h4ec10b9_2    defaults
statsmodels               0.8.0            py27hc87d62d_0    defaults
subprocess32              3.2.7            py27h373dbce_0    defaults
sympy                     1.1.1            py27hc28188a_0    defaults
tblib                     1.3.2            py27h51fe5ba_0    defaults
terminado                 0.8.1                    py27_1    defaults
testpath                  0.3.1            py27hc38d2c4_0    defaults
tk                        8.6.7                hc745277_3    defaults
toolz                     0.9.0                    py27_0    defaults
tornado                   4.5.3                    py27_0    defaults
traitlets                 4.3.2            py27hd6ce930_0    defaults
typing                    3.6.2            py27h66f49e2_0    defaults
unicodecsv                0.14.1           py27h5062da9_0    defaults
unixodbc                  2.3.4                hc36303a_1    defaults
urllib3                   1.22             py27ha55213b_0    defaults
wcwidth                   0.1.7            py27h9e3e1ab_0    defaults
webencodings              0.5.1            py27hff10b21_1    defaults
werkzeug                  0.14.1                   py27_0    defaults
wheel                     0.30.0           py27h2bc6bb2_1    defaults
widgetsnbextension        3.1.0                    py27_0    defaults
wrapt                     1.10.11          py27h04f6869_0    defaults
xlrd                      1.1.0            py27ha77178f_1    defaults
xlsxwriter                1.0.2            py27h12cbc6b_0    defaults
xlwt                      1.3.0            py27h3d85d97_0    defaults
xz                        5.2.3                h55aa19d_2    defaults
yaml                      0.1.7                had09818_2    defaults
zeromq                    4.2.2                hbedb6e5_2    defaults
zict                      0.1.3            py27h12c336c_0    defaults
zlib                      1.2.11               ha838bed_2    defaults
mazerj commented 6 years ago

Just discovered another example of the weakref problem. The following is from a fresh install and update of stock anaconda2 python 2.7 (installed and then conda update --all; conda install virtualenv):

[#2] mazer@glacier $ ~/anaconda2/bin/virtualenv env
New python executable in /auto/share/src/pype3/env/bin/python
Installing setuptools, pip, wheel...
  Complete output from command /auto/share/src/pype3/env/bin/python - setuptools pip wheel:
  Traceback (most recent call last):
  File "<stdin>", line 11, in <module>
  File "/auto/home/mazer/anaconda2/lib/python2.7/site-packages/virtualenv_support/pip-9.0.3-py2.py3-none-any.whl/pip/__init__.py", line 5, in <module>
  File "/auto/home/mazer/anaconda2/lib/python2.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, cStringIO, traceback, warnings, weakref, collections
  File "/auto/home/mazer/anaconda2/lib/python2.7/weakref.py", line 14, in <module>
    from _weakref import (
ImportError: cannot import name _remove_dead_weakref
----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
  File "/auto/home/mazer/anaconda2/bin/virtualenv", line 11, in <module>
    sys.exit(main())
  File "/auto/home/mazer/anaconda2/lib/python2.7/site-packages/virtualenv.py", line 712, in main
    symlink=options.symlink)
  File "/auto/home/mazer/anaconda2/lib/python2.7/site-packages/virtualenv.py", line 953, in create_environment
    download=download,
  File "/auto/home/mazer/anaconda2/lib/python2.7/site-packages/virtualenv.py", line 904, in install_wheel
    call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
  File "/auto/home/mazer/anaconda2/lib/python2.7/site-packages/virtualenv.py", line 796, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /auto/share/src/pype3/env/bin/python - setuptools pip wheel failed with error code 1

I'm thinking that somehow the anaconda binary is linking against system libs even though LD_LIBRARY is empty, though for the life of my I can't figure out where or reliably predict when it's going to happen. Again, this is a pretty much stock Mint 18.2 installation (same as initial post above).

zachlowry commented 5 years ago

I see the same behaviour whenever setting the SUID bit or when adding capabilities to the Python binary. You're correct that the library search path is being overridden with the system path. In fact, it loads the system Python interpreter when executing the anaconda wrapper application because that's the libpython.so that it finds.

mazerj commented 5 years ago

Actually, same here -- I originally came across the problem with a more complicated suid-root script I wrote to start a real-time data collection process, but this one seemed easier to post so others could replicate.. So, is this a bug? I would have thought a bug preventing installation of virtualenv would have set off alarm bells all over the place :-)