search

conda-forge / htcondor-feedstock

A conda-smithy repository for htcondor.
BSD 3-Clause "New" or "Revised" License
0 stars 7 forks source link

condor_adstash script is packaged, but not the adstash Python library #220

Closed jasoncpatton closed 7 months ago

jasoncpatton commented 8 months ago

Solution to issue cannot be found in the documentation.

Issue

I had a potential condor_adstash user write to me asking about an error like this:

(htcondor) [root@3a27166c488a ~]# condor_adstash
Traceback (most recent call last):
  File "/root/miniconda3/envs/htcondor/bin/condor_adstash", line 32, in <module>
    from adstash.adstash import adstash
ModuleNotFoundError: No module named 'adstash'

It turns out that whlie we package the condor_adstash script, we do not package the adstash library it depends upon (which is typically loaded from $(condor_config_val LIBEXEC)/adstash).

Output below is from quickly reproducing this in a barebones container with miniconda installed (hence the /root-based environment).

Installed packages

# Name                    Version                   Build  Channel                                                                                                                                        
_libgcc_mutex             0.1                 conda_forge    conda-forge                                                                                                                                  
_openmp_mutex             4.5                       2_gnu    conda-forge                                                                                                                                  
bzip2                     1.0.8                hd590300_5    conda-forge                                                                                                                                  
c-ares                    1.27.0               hd590300_0    conda-forge                                                                                                                                  
ca-certificates           2024.2.2             hbcca054_0    conda-forge                                                                                                                                  
htcondor-classads         23.4.0               h1bc8f3f_0    conda-forge                                                                                                                                  
htcondor-utils            23.4.0               h841b71b_0    conda-forge                                                                                                                                  
keyutils                  1.6.1                h166bdaf_0    conda-forge                                                                                                                                  
krb5                      1.21.2               h659d440_0    conda-forge                                                                                                                                  
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge                                                                                                                                  
libblas                   3.9.0           21_linux64_openblas    conda-forge                                                                                                                              
libboost-python           1.82.0          py312hfb10629_6    conda-forge                                                                                                                                  
libcblas                  3.9.0           21_linux64_openblas    conda-forge                                                                                                                              
libcondor_utils           23.4.0               hf167c69_0    conda-forge                                                                                                                                  
libcurl                   8.5.0                hca28451_0    conda-forge                                                                                                                                  
libedit                   3.1.20191231         he28a2e2_2    conda-forge                                                                                                                                  
libev                     4.33                 hd590300_2    conda-forge                                                                                                                                  
libexpat                  2.5.0                hcb278e6_1    conda-forge                                                                                                                                  
libffi                    3.4.2                h7f98852_5    conda-forge                                                                                                                                  
libgcc-ng                 13.2.0               h807b86a_5    conda-forge
libgfortran-ng            13.2.0               h69a702a_5    conda-forge
libgfortran5              13.2.0               ha4646dd_5    conda-forge
libgomp                   13.2.0               h807b86a_5    conda-forge
liblapack                 3.9.0           21_linux64_openblas    conda-forge
libnghttp2                1.58.0               h47da74e_1    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.26          pthreads_h413a1c8_0    conda-forge
libsqlite                 3.45.1               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_5    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
munge                     0.5.13               h1c5bbd1_1    conda-forge
ncurses                   6.4                  h59595ed_2    conda-forge
numpy                     1.26.4          py312heda63a1_0    conda-forge
openssl                   3.2.1                hd590300_0    conda-forge
pcre2                     10.42                hcad00b1_0    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
python                    3.12.2          hab00c5b_0_cpython    conda-forge
python-htcondor           23.4.0          py312he90800d_0    conda-forge
python_abi                3.12                    4_cp312    conda-forge
readline                  8.2                  h8228510_1    conda-forge
scitokens-cpp             1.0.2                haea88ab_0    conda-forge
setuptools                69.1.1             pyhd8ed1ab_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

Environment info

active environment : htcondor
    active env location : /root/miniconda3/envs/htcondor
            shell level : 2
       user config file : /root/.condarc
 populated config files : 
          conda version : 23.11.0
    conda-build version : not installed
         python version : 3.11.5.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=cascadelake
                          __conda=23.11.0=0
                          __glibc=2.34=0
                          __linux=4.18.0=0
                          __unix=0=0
       base environment : /root/miniconda3  (writable)
      conda av data dir : /root/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /root/miniconda3/pkgs
                          /root/.conda/pkgs
       envs directories : /root/miniconda3/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/23.11.0 requests/2.31.0 CPython/3.11.5 Linux/4.18.0-517.el8.x86_64 almalinux/9.3 glibc/2.34 solver/libmamba conda-libmamba-solver/23.12.0 libmambapy/1.5.3
                UID:GID : 0:0
             netrc file : None
           offline mode : False
jasoncpatton commented 8 months ago

@duncanmmacleod two ways of solving this, I think:

  1. Follow what we do in the RPMs and stick the src/condor_scripts/adstash directory in $CONDA_PREFIX/libexec, and then update LIBEXEC in the condor config we ship.
  2. Stick adstash (from src/condor_scripts/adstash) in $CONDA_PREFIX/lib/python3.12/site-packages, which seems a little dirty since it's not really a separate package and has no package metadata...
jasoncpatton commented 8 months ago

Or perhaps even 3. Make a new htcondor-adstash package in conda-forge.

jasoncpatton commented 8 months ago

I'm about to be on an extended leave and this is relatively low priority compared to other efforts that are needed, relinquishing ownership for now...