MatterMiners / tardis

Transparent Adaptive Resource Dynamic Integration System
https://cobald-tardis.readthedocs.io
MIT License
15 stars 20 forks source link

pip istall fails #324

Closed rodwalker closed 6 months ago

rodwalker commented 6 months ago

Hi, I tried python3 -m pip install cobald-tardis on centos7 but it fails

...
Collecting python-auditor==0.1.0 (from cobald-tardis)
  Using cached https://files.pythonhosted.org/packages/5c/e9/e1abd31521351eaf378ad366ddf8f3ef7b50e6ae87e46fc966825407c861/python_auditor-0.1.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/lib64/python3.6/tokenize.py", line 452, in open
        buffer = _builtin_open(filename, 'rb')
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-build-ui0f79uy/python-auditor/setup.py'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-ui0f79uy/python-auditor/

$ python3 --version
Python 3.6.8

It is worse on alma9
$ python3 -m pip install cobald-tardis
Defaulting to user installation because normal site-packages is not writeable
Collecting cobald-tardis
  Downloading cobald-tardis-0.8.1.tar.gz (82 kB)
     |████████████████████████████████| 82 kB 2.1 MB/s 
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/pip/_internal/utils/unpacking.py", line 198, in untar_file
    tarfile.data_filter(member.replace(name=fn), location)
AttributeError: module 'tarfile' has no attribute 'data_filter'

During handling of the above exception, another exception occurred:

$ python3 --version Python 3.9.16

Cheers, Rod.

QuantumDancer commented 6 months ago

Hi Rod,

cobald/tardis has dropped support for python3.6 and python3.7 some time ago. So I think what you are installing on centos7 is an old version of cobald/tardis. I will need to look closer why the python-auditor installation fails, but please not that this is version 0.1.0, while the latest one is 0.3.1.

Regarding alma9: At least from a clean alma9 docker image I can install cobald-tardis:

$ docker run -it --rm almalinux:9.3 /bin/bash
$ yum install -y -q  python3-pip 
$ pip install wheel  # also works without

$ pip --version
pip 21.2.3 from /usr/lib/python3.9/site-packages/pip (python 3.9)

$ pip install cobald-tardis

$ cobald -h
usage: cobald [-h] [--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--log-target LOG_TARGET] [--log-journal] CONFIGURATION

COBalD - the Opportunistic Balancing Daemon

positional arguments:
  CONFIGURATION         path of the configuration to use

optional arguments:
  -h, --help            show this help message and exit

Startup Logging:
  --log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        initial logging level
  --log-target LOG_TARGET
                        initial logging target; stdout, stderr or a file path
  --log-journal         use short formatting suitable for journals

Looking at your error message, it seems that there is a problem with python3/pip installation, as it cannot find the function data_filter for the tarfile module:

File "/usr/lib/python3.9/site-packages/pip/_internal/utils/unpacking.py", line 198, in untar_file
tarfile.data_filter(member.replace(name=fn), location)
AttributeError: module 'tarfile' has no attribute 'data_filter'

Could you check if there is something wrong with the tarfile module? This is how it looks for me:

python3 
Python 3.9.18 (main, Sep  7 2023, 00:00:00) 
[GCC 11.4.1 20230605 (Red Hat 11.4.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tarfile
>>> tarfile.__file__
'/usr/lib64/python3.9/tarfile.py'
>>> help(tarfile.data_filter)

Help on function data_filter in module tarfile:

data_filter(member, dest_path)

In any case, there are also a docker containers provided at https://hub.docker.com/r/matterminers/cobald-tardis This is how we run cobald/tardis in Freiburg.

Cheers, Benjamin

giffels commented 6 months ago

As @QuantumDancer mentioned we dropped support for Python3.6 and Python 3.7 in cobald-tardis:0.8.0. But installing cobald-tardis:0.7.1 still works fine for me on centos 7.

docker run -it centos:7 /bin/bash                                                                                                                                            255
[root@088f06073459 /]# yum install epel-release
Loaded plugins: fastestmirror, ovl
Determining fastest mirrors
 * base: mirror.fra1.de.leaseweb.net
 * extras: mirror.fra1.de.leaseweb.net
 * updates: mirror1.hs-esslingen.de
base                                                                                                                                                                                                  | 3.6 kB  00:00:00
extras                                                                                                                                                                                                | 2.9 kB  00:00:00
updates                                                                                                                                                                                               | 2.9 kB  00:00:00
(1/4): base/7/x86_64/group_gz                                                                                                                                                                         | 153 kB  00:00:00
(2/4): extras/7/x86_64/primary_db                                                                                                                                                                     | 250 kB  00:00:00
(3/4): updates/7/x86_64/primary_db                                                                                                                                                                    |  24 MB  00:00:01
(4/4): base/7/x86_64/primary_db                                                                                                                                                                       | 6.1 MB  00:00:01
Resolving Dependencies
--> Running transaction check
---> Package epel-release.noarch 0:7-11 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================================================================================
 Package                                                   Arch                                                Version                                             Repository                                           Size
=============================================================================================================================================================================================================================
Installing:
 epel-release                                              noarch                                              7-11                                                extras                                               15 k

Transaction Summary
=============================================================================================================================================================================================================================
Install  1 Package

Total download size: 15 k
Installed size: 24 k
Is this ok [y/d/N]: y
Downloading packages:
warning: /var/cache/yum/x86_64/7/extras/packages/epel-release-7-11.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
Public key for epel-release-7-11.noarch.rpm is not installed
epel-release-7-11.noarch.rpm                                                                                                                                                                          |  15 kB  00:00:00
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
 Userid     : "CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>"
 Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
 Package    : centos-release-7-9.2009.0.el7.centos.x86_64 (@CentOS)
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Is this ok [y/N]: y
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : epel-release-7-11.noarch                                                                                                                                                                                  1/1
  Verifying  : epel-release-7-11.noarch                                                                                                                                                                                  1/1

Installed:
  epel-release.noarch 0:7-11

Complete!
[root@088f06073459 /]# yum install python36 python36-dev python36-pip
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
 * base: mirror.fra1.de.leaseweb.net
 * epel: cdn.centos.no
 * extras: mirror.fra1.de.leaseweb.net
 * updates: mirror1.hs-esslingen.de
No package python36-dev available.
Resolving Dependencies
--> Running transaction check
---> Package python3.x86_64 0:3.6.8-21.el7_9 will be installed
--> Processing Dependency: python3-libs(x86-64) = 3.6.8-21.el7_9 for package: python3-3.6.8-21.el7_9.x86_64
--> Processing Dependency: python3-setuptools for package: python3-3.6.8-21.el7_9.x86_64
--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-21.el7_9.x86_64
---> Package python3-pip.noarch 0:9.0.3-8.el7 will be installed
--> Running transaction check
---> Package python3-libs.x86_64 0:3.6.8-21.el7_9 will be installed
--> Processing Dependency: libtirpc.so.1()(64bit) for package: python3-libs-3.6.8-21.el7_9.x86_64
---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed
--> Running transaction check
---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================================================================================
 Package                                                     Arch                                            Version                                                  Repository                                        Size
=============================================================================================================================================================================================================================
Installing:
 python3                                                     x86_64                                          3.6.8-21.el7_9                                           updates                                           71 k
 python3-pip                                                 noarch                                          9.0.3-8.el7                                              base                                             1.6 M
Installing for dependencies:
 libtirpc                                                    x86_64                                          0.2.4-0.16.el7                                           base                                              89 k
 python3-libs                                                x86_64                                          3.6.8-21.el7_9                                           updates                                          7.0 M
 python3-setuptools                                          noarch                                          39.2.0-10.el7                                            base                                             629 k

Transaction Summary
=============================================================================================================================================================================================================================
Install  2 Packages (+3 Dependent packages)

Total download size: 9.3 M
Installed size: 48 M
Is this ok [y/d/N]: y
Downloading packages:
(1/5): libtirpc-0.2.4-0.16.el7.x86_64.rpm                                                                                                                                                             |  89 kB  00:00:00
(2/5): python3-3.6.8-21.el7_9.x86_64.rpm                                                                                                                                                              |  71 kB  00:00:00
(3/5): python3-setuptools-39.2.0-10.el7.noarch.rpm                                                                                                                                                    | 629 kB  00:00:00
(4/5): python3-pip-9.0.3-8.el7.noarch.rpm                                                                                                                                                             | 1.6 MB  00:00:00
(5/5): python3-libs-3.6.8-21.el7_9.x86_64.rpm                                                                                                                                                         | 7.0 MB  00:00:00
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                         10 MB/s | 9.3 MB  00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                            1/5
  Installing : python3-setuptools-39.2.0-10.el7.noarch                                                                                                                                                                   2/5
  Installing : python3-pip-9.0.3-8.el7.noarch                                                                                                                                                                            3/5
  Installing : python3-3.6.8-21.el7_9.x86_64                                                                                                                                                                             4/5
  Installing : python3-libs-3.6.8-21.el7_9.x86_64                                                                                                                                                                        5/5
  Verifying  : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                            1/5
  Verifying  : python3-libs-3.6.8-21.el7_9.x86_64                                                                                                                                                                        2/5
  Verifying  : python3-3.6.8-21.el7_9.x86_64                                                                                                                                                                             3/5
  Verifying  : python3-setuptools-39.2.0-10.el7.noarch                                                                                                                                                                   4/5
  Verifying  : python3-pip-9.0.3-8.el7.noarch                                                                                                                                                                            5/5

Installed:
  python3.x86_64 0:3.6.8-21.el7_9                                                                              python3-pip.noarch 0:9.0.3-8.el7

Dependency Installed:
  libtirpc.x86_64 0:0.2.4-0.16.el7                                     python3-libs.x86_64 0:3.6.8-21.el7_9                                     python3-setuptools.noarch 0:39.2.0-10.el7

Complete!
[root@088f06073459 /]# pip3 install --upgrade pip
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting pip
  Downloading https://files.pythonhosted.org/packages/a4/6d/6463d49a933f547439d6b5b98b46af8742cc03ae83543e4d7688c2420f8b/pip-21.3.1-py3-none-any.whl (1.7MB)
    100% |################################| 1.7MB 602kB/s
Installing collected packages: pip
Successfully installed pip-21.3.1
[root@088f06073459 /]# pip3 install cobald-tardis
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Collecting cobald-tardis
  Downloading cobald-tardis-0.7.1.tar.gz (85 kB)
     |################################| 85 kB 1.1 MB/s
  Preparing metadata (setup.py) ... done
Collecting cryptography<3.2
  Downloading cryptography-3.1.1-cp35-abi3-manylinux2010_x86_64.whl (2.6 MB)
     |################################| 2.6 MB 3.9 MB/s
Collecting CloudStackAIO>=0.0.8
  Downloading CloudStackAIO-0.0.8.tar.gz (8.7 kB)
  Preparing metadata (setup.py) ... done
Collecting PyYAML
  Downloading PyYAML-6.0.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (677 kB)
     |################################| 677 kB 25.9 MB/s
Collecting AsyncOpenStackClient>=0.9.0
  Downloading AsyncOpenStackClient-0.9.0.tar.gz (25 kB)
  Preparing metadata (setup.py) ... done
Collecting cobald>=0.12.3
  Downloading cobald-0.13.0-py3-none-any.whl (47 kB)
     |################################| 47 kB 3.9 MB/s
Collecting asyncssh
  Downloading asyncssh-2.14.1-py3-none-any.whl (352 kB)
     |################################| 352 kB 1.1 MB/s
Collecting aiotelegraf
  Downloading aiotelegraf-0.4.0-py3-none-any.whl (4.0 kB)
Collecting elasticsearch<8.0.0,>=7.17
  Downloading elasticsearch-7.17.9-py2.py3-none-any.whl (385 kB)
     |################################| 385 kB 24.7 MB/s
Collecting aioprometheus>=21.9.0
  Downloading aioprometheus-22.5.0-py3-none-any.whl (31 kB)
Collecting kubernetes_asyncio
  Downloading kubernetes_asyncio-28.2.1-py3-none-any.whl (2.0 MB)
     |################################| 2.0 MB 5.6 MB/s
Collecting pydantic
  Downloading pydantic-1.9.2-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.2 MB)
     |################################| 11.2 MB 842 kB/s
Collecting asyncstdlib
  Downloading asyncstdlib-3.10.9-py3-none-any.whl (37 kB)
Collecting typing_extensions
  Downloading typing_extensions-4.1.1-py3-none-any.whl (26 kB)
Collecting backports.cached_property
  Downloading backports.cached_property-1.0.2-py3-none-any.whl (6.1 kB)
Collecting python-auditor==0.1.0
  Downloading python_auditor-0.1.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.3 MB)
     |################################| 3.3 MB 9.1 MB/s
Collecting pytz
  Downloading pytz-2023.3.post1-py2.py3-none-any.whl (502 kB)
     |################################| 502 kB 20.4 MB/s
Collecting tzlocal
  Downloading tzlocal-4.2-py3-none-any.whl (19 kB)
Collecting aiolancium
  Downloading aiolancium-0.2.2-py3-none-any.whl (26 kB)
Collecting fastapi-jwt-auth
  Downloading fastapi_jwt_auth-0.5.0-py3-none-any.whl (12 kB)
Collecting python-jose
  Downloading python_jose-3.3.0-py2.py3-none-any.whl (33 kB)
Collecting uvicorn[standard]<=0.14.0
  Downloading uvicorn-0.14.0-py3-none-any.whl (50 kB)
     |################################| 50 kB 5.3 MB/s
Collecting typer
  Downloading typer-0.9.0-py3-none-any.whl (45 kB)
     |################################| 45 kB 3.2 MB/s
Collecting bcrypt
  Downloading bcrypt-4.0.1-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (593 kB)
     |################################| 593 kB 20.4 MB/s
Collecting python-multipart
  Downloading python-multipart-0.0.5.tar.gz (32 kB)
  Preparing metadata (setup.py) ... done
Collecting aiohttp<4.0
  Downloading aiohttp-3.8.6-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (985 kB)
     |################################| 985 kB 17.9 MB/s
Collecting fastapi<0.84.0
  Downloading fastapi-0.83.0-py3-none-any.whl (55 kB)
     |################################| 55 kB 3.8 MB/s
Collecting aiosignal>=1.1.2
  Downloading aiosignal-1.2.0-py3-none-any.whl (8.2 kB)
Collecting idna-ssl>=1.0
  Downloading idna-ssl-1.1.0.tar.gz (3.4 kB)
  Preparing metadata (setup.py) ... done
Collecting yarl<2.0,>=1.0
  Downloading yarl-1.7.2-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (270 kB)
     |################################| 270 kB 7.7 MB/s
Collecting attrs>=17.3.0
  Downloading attrs-22.2.0-py3-none-any.whl (60 kB)
     |################################| 60 kB 4.0 MB/s
Collecting asynctest==0.13.0
  Downloading asynctest-0.13.0-py3-none-any.whl (26 kB)
Collecting async-timeout<5.0,>=4.0.0a3
  Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting frozenlist>=1.1.1
  Downloading frozenlist-1.2.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (191 kB)
     |################################| 191 kB 6.0 MB/s
Collecting multidict<7.0,>=4.5
  Downloading multidict-5.2.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (159 kB)
     |################################| 159 kB 7.4 MB/s
Collecting charset-normalizer<4.0,>=2.0
  Downloading charset_normalizer-3.0.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (162 kB)
     |################################| 162 kB 20.6 MB/s
Collecting quantile-python>=1.1
  Downloading quantile-python-1.1.tar.gz (2.9 kB)
  Preparing metadata (setup.py) ... done
Collecting orjson
  Downloading orjson-3.6.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (233 kB)
     |################################| 233 kB 17.0 MB/s
Collecting simplejson==3.16.0
  Downloading simplejson-3.16.0.tar.gz (81 kB)
     |################################| 81 kB 5.8 MB/s
  Preparing metadata (setup.py) ... done
Collecting simple-rest-client==0.5.4
  Downloading simple_rest_client-0.5.4-py3-none-any.whl (6.7 kB)
Collecting python-dateutil==2.8.0
  Downloading python_dateutil-2.8.0-py2.py3-none-any.whl (226 kB)
     |################################| 226 kB 12.2 MB/s
Collecting six>=1.5
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting python-status>=1.0.1
  Downloading python-status-1.0.1.tar.gz (3.8 kB)
  Preparing metadata (setup.py) ... done
Collecting requests>=2.13.0
  Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB)
     |################################| 63 kB 2.1 MB/s
Collecting json-encoder>=0.4.4
  Downloading json_encoder-0.4.4-py2.py3-none-any.whl (7.4 kB)
Collecting toposort
  Downloading toposort-1.10-py3-none-any.whl (8.5 kB)
Collecting trio>=0.4.0
  Downloading trio-0.19.0-py3-none-any.whl (356 kB)
     |################################| 356 kB 1.0 MB/s
Collecting entrypoints
  Downloading entrypoints-0.4-py3-none-any.whl (5.3 kB)
Collecting cffi!=1.11.3,>=1.8
  Downloading cffi-1.15.1-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl (402 kB)
     |################################| 402 kB 4.5 MB/s
Collecting certifi
  Downloading certifi-2023.11.17-py3-none-any.whl (162 kB)
     |################################| 162 kB 6.7 MB/s
Collecting urllib3<2,>=1.21.1
  Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)
     |################################| 143 kB 21.1 MB/s
Collecting starlette==0.19.1
  Downloading starlette-0.19.1-py3-none-any.whl (63 kB)
     |################################| 63 kB 2.2 MB/s
Collecting contextlib2>=21.6.0
  Downloading contextlib2-21.6.0-py2.py3-none-any.whl (13 kB)
Collecting anyio<5,>=3.4.0
  Downloading anyio-3.6.2-py3-none-any.whl (80 kB)
     |################################| 80 kB 6.3 MB/s
Collecting dataclasses>=0.6
  Downloading dataclasses-0.8-py3-none-any.whl (19 kB)
Collecting h11>=0.8
  Downloading h11-0.13.0-py3-none-any.whl (58 kB)
     |################################| 58 kB 5.0 MB/s
Collecting asgiref>=3.3.4
  Downloading asgiref-3.4.1-py3-none-any.whl (25 kB)
Collecting click>=7.*
  Downloading click-8.0.4-py3-none-any.whl (97 kB)
     |################################| 97 kB 7.0 MB/s
Collecting python-dotenv>=0.13
  Downloading python_dotenv-0.20.0-py3-none-any.whl (17 kB)
Collecting websockets>=9.1
  Downloading websockets-9.1-cp36-cp36m-manylinux2010_x86_64.whl (102 kB)
     |################################| 102 kB 8.3 MB/s
Collecting httptools==0.2.*
  Downloading httptools-0.2.0-cp36-cp36m-manylinux1_x86_64.whl (343 kB)
     |################################| 343 kB 18.3 MB/s
Collecting watchgod>=0.6
  Downloading watchgod-0.7-py3-none-any.whl (11 kB)
Collecting uvloop!=0.15.0,!=0.15.1,>=0.14.0
  Downloading uvloop-0.14.0-cp36-cp36m-manylinux2010_x86_64.whl (3.9 MB)
     |################################| 3.9 MB 18.9 MB/s
Collecting pyjwt<=2.4.0
  Downloading PyJWT-2.4.0-py3-none-any.whl (18 kB)
Collecting jsonref<2.0.0,>=1.0.0
  Downloading jsonref-1.0.1-py3-none-any.whl (9.5 kB)
Collecting pytelegraf>=0.3.3
  Downloading pytelegraf-0.3.3.tar.gz (6.8 kB)
  Preparing metadata (setup.py) ... done
Collecting asyncssh
  Downloading asyncssh-2.14.0-py3-none-any.whl (351 kB)
     |################################| 351 kB 7.9 MB/s
  Downloading asyncssh-2.13.2-py3-none-any.whl (349 kB)
     |################################| 349 kB 17.9 MB/s
Collecting typing>=3.6
  Downloading typing-3.7.4.3.tar.gz (78 kB)
     |################################| 78 kB 5.5 MB/s
  Preparing metadata (setup.py) ... done
Collecting pyjwt<=2.4.0
  Downloading PyJWT-1.7.1-py2.py3-none-any.whl (18 kB)
Requirement already satisfied: setuptools>=21.0.0 in /usr/lib/python3.6/site-packages (from kubernetes_asyncio->cobald-tardis) (39.2.0)
Collecting pyasn1
  Downloading pyasn1-0.5.1-py2.py3-none-any.whl (84 kB)
     |################################| 84 kB 778 kB/s
Collecting rsa
  Downloading rsa-4.9-py3-none-any.whl (34 kB)
Collecting ecdsa!=0.15
  Downloading ecdsa-0.18.0-py2.py3-none-any.whl (142 kB)
     |################################| 142 kB 9.2 MB/s
Collecting backports.zoneinfo
  Downloading backports.zoneinfo-0.2.1-cp36-cp36m-manylinux1_x86_64.whl (70 kB)
     |################################| 70 kB 6.5 MB/s
Collecting pytz-deprecation-shim
  Downloading pytz_deprecation_shim-0.1.0.post0-py2.py3-none-any.whl (15 kB)
Collecting pycparser
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     |################################| 118 kB 9.6 MB/s
Collecting importlib-metadata
  Downloading importlib_metadata-4.8.3-py3-none-any.whl (17 kB)
Collecting idna>=2.0
  Downloading idna-3.6-py3-none-any.whl (61 kB)
     |################################| 61 kB 139 kB/s
Collecting outcome
  Downloading outcome-1.1.0-py2.py3-none-any.whl (9.7 kB)
Collecting async-generator>=1.9
  Downloading async_generator-1.10-py3-none-any.whl (18 kB)
Collecting contextvars>=2.1
  Downloading contextvars-2.4.tar.gz (9.6 kB)
  Preparing metadata (setup.py) ... done
Collecting sortedcontainers
  Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting sniffio
  Downloading sniffio-1.2.0-py3-none-any.whl (10 kB)
Collecting importlib-resources
  Downloading importlib_resources-5.4.0-py3-none-any.whl (28 kB)
Collecting tzdata
  Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB)
     |################################| 341 kB 16.4 MB/s
Collecting immutables>=0.9
  Downloading immutables-0.19-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (116 kB)
     |################################| 116 kB 28.1 MB/s
Collecting singledispatch>=3.4.0.3
  Downloading singledispatch-3.7.0-py2.py3-none-any.whl (9.2 kB)
Collecting charset-normalizer<4.0,>=2.0
  Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting zipp>=0.5
  Downloading zipp-3.6.0-py3-none-any.whl (5.3 kB)
Using legacy 'setup.py install' for cobald-tardis, since package 'wheel' is not installed.
Using legacy 'setup.py install' for AsyncOpenStackClient, since package 'wheel' is not installed.
Using legacy 'setup.py install' for simplejson, since package 'wheel' is not installed.
Using legacy 'setup.py install' for CloudStackAIO, since package 'wheel' is not installed.
Using legacy 'setup.py install' for python-multipart, since package 'wheel' is not installed.
Using legacy 'setup.py install' for idna-ssl, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pytelegraf, since package 'wheel' is not installed.
Using legacy 'setup.py install' for quantile-python, since package 'wheel' is not installed.
Using legacy 'setup.py install' for typing, since package 'wheel' is not installed.
Using legacy 'setup.py install' for contextvars, since package 'wheel' is not installed.
Using legacy 'setup.py install' for python-status, since package 'wheel' is not installed.
Installing collected packages: typing-extensions, immutables, contextvars, zipp, sniffio, six, multidict, idna, frozenlist, dataclasses, yarl, urllib3, singledispatch, pycparser, importlib-resources, importlib-metadata, idna-ssl, contextlib2, charset-normalizer, certifi, attrs, asynctest, async-timeout, anyio, aiosignal, tzdata, starlette, sortedcontainers, requests, python-status, pydantic, pyasn1, outcome, json-encoder, h11, click, cffi, backports.zoneinfo, async-generator, asgiref, aiohttp, websockets, watchgod, uvloop, uvicorn, typing, trio, toposort, simplejson, simple-rest-client, rsa, quantile-python, PyYAML, pytz-deprecation-shim, python-dotenv, python-dateutil, pytelegraf, pyjwt, orjson, jsonref, httptools, fastapi, entrypoints, ecdsa, cryptography, tzlocal, typer, pytz, python-multipart, python-jose, python-auditor, kubernetes-asyncio, fastapi-jwt-auth, elasticsearch, cobald, CloudStackAIO, bcrypt, backports.cached-property, asyncstdlib, asyncssh, AsyncOpenStackClient, aiotelegraf, aioprometheus, aiolancium, cobald-tardis
    Running setup.py install for contextvars ... done
    Running setup.py install for idna-ssl ... done
    Running setup.py install for python-status ... done
    Running setup.py install for typing ... done
    Running setup.py install for simplejson ... done
    Running setup.py install for quantile-python ... done
    Running setup.py install for pytelegraf ... done
    Running setup.py install for python-multipart ... done
    Running setup.py install for CloudStackAIO ... done
    Running setup.py install for AsyncOpenStackClient ... done
    Running setup.py install for cobald-tardis ... done
Successfully installed AsyncOpenStackClient-0.9.0 CloudStackAIO-0.0.8 PyYAML-6.0.1 aiohttp-3.8.6 aiolancium-0.2.2 aioprometheus-22.5.0 aiosignal-1.2.0 aiotelegraf-0.4.0 anyio-3.6.2 asgiref-3.4.1 async-generator-1.10 async-timeout-4.0.2 asyncssh-2.13.2 asyncstdlib-3.10.9 asynctest-0.13.0 attrs-22.2.0 backports.cached-property-1.0.2 backports.zoneinfo-0.2.1 bcrypt-4.0.1 certifi-2023.11.17 cffi-1.15.1 charset-normalizer-2.0.12 click-8.0.4 cobald-0.13.0 cobald-tardis-0.7.1 contextlib2-21.6.0 contextvars-2.4 cryptography-3.1.1 dataclasses-0.8 ecdsa-0.18.0 elasticsearch-7.17.9 entrypoints-0.4 fastapi-0.83.0 fastapi-jwt-auth-0.5.0 frozenlist-1.2.0 h11-0.13.0 httptools-0.2.0 idna-3.6 idna-ssl-1.1.0 immutables-0.19 importlib-metadata-4.8.3 importlib-resources-5.4.0 json-encoder-0.4.4 jsonref-1.0.1 kubernetes-asyncio-28.2.1 multidict-5.2.0 orjson-3.6.1 outcome-1.1.0 pyasn1-0.5.1 pycparser-2.21 pydantic-1.9.2 pyjwt-1.7.1 pytelegraf-0.3.3 python-auditor-0.1.0 python-dateutil-2.8.0 python-dotenv-0.20.0 python-jose-3.3.0 python-multipart-0.0.5 python-status-1.0.1 pytz-2023.3.post1 pytz-deprecation-shim-0.1.0.post0 quantile-python-1.1 requests-2.27.1 rsa-4.9 simple-rest-client-0.5.4 simplejson-3.16.0 singledispatch-3.7.0 six-1.16.0 sniffio-1.2.0 sortedcontainers-2.4.0 starlette-0.19.1 toposort-1.10 trio-0.19.0 typer-0.9.0 typing-3.7.4.3 typing-extensions-4.1.1 tzdata-2023.3 tzlocal-4.2 urllib3-1.26.18 uvicorn-0.14.0 uvloop-0.14.0 watchgod-0.7 websockets-9.1 yarl-1.7.2 zipp-3.6.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[root@088f06073459 /]# cobald --help
usage: cobald [-h] [--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
              [--log-target LOG_TARGET] [--log-journal]
              CONFIGURATION

COBalD - the Opportunistic Balancing Daemon

positional arguments:
  CONFIGURATION         path of the configuration to use

optional arguments:
  -h, --help            show this help message and exit

Startup Logging:
  --log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        initial logging level
  --log-target LOG_TARGET
                        initial logging target; stdout, stderr or a file path
  --log-journal         use short formatting suitable for journals
rodwalker commented 6 months ago

Hi,

python3

Python 3.9.16 (main, Sep 12 2023, 00:00:00) [GCC 11.3.1 20221121 (Red Hat 11.3.1-4)] on linux Type "help", "copyright", "credits" or "license" for more information.

import tarfile tarfile.file '/usr/lib64/python3.9/tarfile.py' help(tarfile.data_filter) Traceback (most recent call last): File "", line 1, in AttributeError: module 'tarfile' has no attribute 'data_filter'

Minor version difference in th python version. Updating to Python 3.9.18 worked.

Interestingly it worked on alma9 in a venv

@. ~]$ python3 -m venv ct @. ~]$ cd ct/ @.*** ct]$ . bin/activate

I am using docker but would need slurm client config, with munge key, so maybe better to do this myself.

BTW - this is about a week long DT for the LRZ storage from Monday. maybe we can run using KIT RSE via CT.

Cheers, Rod.

On Tue, 12 Dec 2023 at 13:18, Benjamin Rottler @.***> wrote:

Hi Rod,

cobald/tardis has dropped support for python3.6 and python3.7 some time ago. So I think what you are installing on centos7 is an old version of cobald/tardis. I will need to look closer why the python-auditor installation fails, but please not that this is version 0.1.0, while the latest one is 0.3.1.

Regarding alma9: At least from a clean alma9 docker image I can install cobald-tardis:

$ docker run -it --rm almalinux:9.3 /bin/bash $ yum install -y -q python3-pip $ pip install wheel # also works without

$ pip --version pip 21.2.3 from /usr/lib/python3.9/site-packages/pip (python 3.9)

$ pip install cobald-tardis

$ cobald -h usage: cobald [-h] [--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--log-target LOG_TARGET] [--log-journal] CONFIGURATION

COBalD - the Opportunistic Balancing Daemon

positional arguments: CONFIGURATION path of the configuration to use

optional arguments: -h, --help show this help message and exit

Startup Logging: --log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL} initial logging level --log-target LOG_TARGET initial logging target; stdout, stderr or a file path --log-journal use short formatting suitable for journals

Looking at your error message, it seems that there is a problem with python3/pip installation, as it cannot find the function data_filter for the tarfile module:

File "/usr/lib/python3.9/site-packages/pip/_internal/utils/unpacking.py", line 198, in untar_file tarfile.data_filter(member.replace(name=fn), location) AttributeError: module 'tarfile' has no attribute 'data_filter'

Could you check if there is something wrong with the tarfile module? This is how it looks for me:

python3 Python 3.9.18 (main, Sep 7 2023, 00:00:00) [GCC 11.4.1 20230605 (Red Hat 11.4.1-2)] on linux Type "help", "copyright", "credits" or "license" for more information.

import tarfile tarfile.file '/usr/lib64/python3.9/tarfile.py' help(tarfile.data_filter)

Help on function data_filter in module tarfile:

data_filter(member, dest_path)

In any case, there are also a docker containers provided at https://hub.docker.com/r/matterminers/cobald-tardis This is how we run cobald/tardis in Freiburg.

Cheers, Benjamin

— Reply to this email directly, view it on GitHub https://github.com/MatterMiners/tardis/issues/324#issuecomment-1851927066, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNFVRQPV3HDIAYNM6VLAPTYJBDQRAVCNFSM6AAAAABARHX66CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJRHEZDOMBWGY . You are receiving this because you authored the thread.Message ID: @.***>

giffels commented 6 months ago

Closing this, since pip installation is working on both centos:7 and almalinux:9.