Closed FernandoDurier closed 3 years ago
What is in your requirements.txt
file?
It seems a mistake to specify ibm_db-0.2.0
because the currrent version is 3.0.4
Hi, @imavo !! Sorry I clipped only the latest part of the log. It was set to any version value. Problem is that since I didn't specified the Python Docker Image tried all the possible version stopping at the oldest. Here is the new requirements.txt
ibm_db==3.0.4
elasticsearch
pandas
sklearn
scikit-learn
flask
flask_cors
gunicorn
joblib
ibm-watson-machine-learning
The problem still remains, though...
If you start a container from the base Python 3.8 image (i.e. without your app and without your requirements.txt), and then shell into that container , can you run pip install ibm_db
on it's own? I suspect your networking configuration has changed somehow, meaning external python packages cannot be installed.
Hello!! Today I got a message from our DBA team here at IBM, and they told me that the license odbc has expired. Since it happened when the issue appeared, I'm suspecting the error may come from it ... because if you check the error log the python lib is complaining about the IBM DataDriver Env not being setup ... since the license has expired maybe it is related. I'm waiting for the new license, then I will test it and get back here ;)
The license expiring will prevent connecting, but it shouldn't prevent installation. I'm having similar problems; an automated build of something that hasn't changed in weeks suddenly started failing a few days ago, due to ibm-db not being installable.
Hello, @Tam-Lin !! Apparently, the ibm_db lib needs the IBM_DB_HOME to be set to be fully installed, so internal scripts execute some kind of validations over the clidriver. My guess is that the expiring licensing is hurting this part of the process. Do you use it dockerized too?
there is a report (see google) that setuptools v58 breaks support for a feature needed by the current ibm_db setup called use_2to3
.
workaround required while waiting for resolution...
@imavo Do you have the link? Or the way around code? I've just googled it here by ("python setuptools v58 breaks ibm db") and didn't get any answer...
Carefully check your log files for references to use_2to3
, and the version of setuptools in use. Check github setuptools issues page for that keyword. ibm_db is not the only package that is impacted by the use_2to3 is invalid
symptom. If root cause is confirmed , I expect workaround or fix to emerge before long.
@imavo : We ran into the same issue. It's the setuptools
that gets upgraded and caused the issue. We rollback to setuptools==50.3.2
to resolve the issue.
Hmm interesting. I might test it out here so. Thank you @sxnguyen !!
Good Lord!! It worked @sxnguyen !!! I've spread the word here at my team.
Again thanks a lot!!
@FernandoDurier @sxnguyen I checked with setuptools 50.3.2
and it's good. I even tried with somewhat latest version i.e. 57.4.0
(July 18 2021) and it is working for us.
That's right. Sorry for the disruption. Setuptools 58 dropped support for the use_2to3
parameter, causing projects like this to install invalid syntax (as reported). Setuptools 58.0.2 fixed the issue by hard failing with an use_2to3 is invalid
message, but that still doesn't address the underlying issue that that usage is unsupported. See #655 and the attached PR that should have fixed the issue.
Hi,
Looks like the PR was not enough to resolve the issue.
For now we have stick to using setuptools version till 57.5.0 or less for the installation of IBM_DB to work fine.
Thanks.
Probably not a big surprise. The PR was merged but (a) no release has been made with the change and (b) I neglected to include MANIFEST.in. I should also note that the fix won't have any impact of setup.py install
is invoked. You should be using pip install
with pip 10 or later (IIRC) to install. But let me see if I can show the fix.
Here it is installing:
ibm_db master $ pwd
/Users/jaraco/p/python-ibmdb/IBM_DB/ibm_db
ibm_db master $ python -m venv env
ibm_db master $ git-id
9b39a75
ibm_db master $ env/bin/pip install .
DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
Processing /Users/jaraco/code/public/python-ibmdb/IBM_DB/ibm_db
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing wheel metadata ... done
Building wheels for collected packages: ibm-db
Building wheel for ibm-db (PEP 517) ... done
Created wheel for ibm-db: filename=ibm_db-3.0.4-cp39-cp39-macosx_11_0_x86_64.whl size=32459073 sha256=6fc3138795b0c07bd41399b54649f452d1c2ea17f1bb32fdaa28c22444ac805b
Stored in directory: /private/var/folders/03/7l0ffypn50b83bp0bt07xcch00n8zm/T/pip-ephem-wheel-cache-2mb3i3uo/wheels/24/c0/04/f01f7809ef854b5c3c3e528dd0be2b9161048d100ff84f9c69
Successfully built ibm-db
Installing collected packages: ibm-db
Successfully installed ibm-db-3.0.4
What's not working for you?
In #655, I illustrate how the proposed fix does address the issue when using pip. I've also confirmed that the change to MANIFEST.in is not needed, as long as Setuptools 43 or later is used to build the sdist.
I created a release and published it to devpi:
And that build includes the pyproject.toml file and installs nicely:
ibm_db master $ pip-run -i https://m.devpi.net/jaraco/dev ibm-db
Looking in indexes: https://m.devpi.net/jaraco/dev
Collecting ibm-db
Downloading https://m.devpi.net/jaraco/dev/%2Bf/0ad/5b913eac1da2d/ibm_db-3.0.4.tar.gz (32.2 MB)
|████████████████████████████████| 32.2 MB 2.1 MB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing wheel metadata ... done
Building wheels for collected packages: ibm-db
Building wheel for ibm-db (PEP 517) ... done
Created wheel for ibm-db: filename=ibm_db-3.0.4-cp310-cp310-macosx_10_15_universal2.whl size=32499082 sha256=d5a9fe5773bfb0cb9de7964269ca263b4c76b100035fbaf1349467ee04ff8c8c
Stored in directory: /Users/jaraco/Library/Caches/pip/wheels/c9/25/ff/83e2e23438320ba61e3c0773c4f551338665dc605bfa5682f3
Successfully built ibm-db
Installing collected packages: ibm-db
Successfully installed ibm-db-3.0.4
Python 3.10.0rc1 (v3.10.0rc1:cc115e5bf6, Aug 2 2021, 17:54:14) [Clang 12.0.5 (clang-1205.0.22.11)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
Though I don't understand why the sdist is 32MB versus the official 3.0.4 release which was about 800k. But since this project does lots of magic with downloading clidriver
, and that content is what's consuming the space, so I'm thinking it's by design.
Hi @jaraco Thanks for #661
I will have to test this change for all platforms and make a new release. Working on the same.
New release made with the support. Thanks @jaraco for the PR.
New version 3.1.0 released.
Please check if this can be closed now.
Here is my Dockerfile: FROM python:3.8 COPY . /app WORKDIR /app RUN cd /app/db2_license/licenses/ && tar -xzf linuxx64_odbc_cli.tar.gz && rm linuxx64_odbc_cli.tar.gz && cd /app RUN pip install --upgrade setuptools pip ENV IBM_DB_HOME /app/db2_license/licenses/clidriver ENV LD_LIBRARY_PATH /app/db2_license/licenses/clidriver/lib:$LD_LIBRARY_PATH RUN pip install -r requirements.txt ENV FLASK_APP app.py CMD [ "gunicorn", "-b", ":5000","--config", "/app/gunicorn_config.py", "app:app"]
Here is the error:
10 19.79 ----------------------------------------
10 19.79 WARNING: Discarding https://files.pythonhosted.org/packages/ba/64/ae0a045a4eb48a839ab2cce39a80416301faf85ff7dfe2d0af2a683eba54/ibm_db-0.2.6.tar.gz#sha256=c3c95f3d882a905840ec9b3f80bf40bbaa09e4d6cbf804362d3131e22e1de632 (from https://pypi.org/simple/ibm-db/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
10 19.83 Downloading ibm_db-0.2.0.tar.gz (157 kB)
10 20.19 ERROR: Command errored out with exit status 1:
10 20.19 command: /usr/local/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-yg73fy3v/ibm-db_bd11f11bf4244e31948879b6c16ab417/setup.py'"'"'; file='"'"'/tmp/pip-install-yg73fy3v/ibm-db_bd11f11bf4244e31948879b6c16ab417/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egginfo --egg-base /tmp/pip-pip-egg-info-pzu4vov
10 20.19 cwd: /tmp/pip-install-yg73fy3v/ibm-db_bd11f11bf4244e31948879b6c16ab417/
10 20.19 Complete output (6 lines):
10 20.19 Traceback (most recent call last):
10 20.19 File "", line 1, in
10 20.19 File "/tmp/pip-install-yg73fy3v/ibm-db_bd11f11bf4244e31948879b6c16ab417/setup.py", line 12
10 20.19 print 'IBM DataServer environment not set. Please refer to the README for more information on building the driver'
10 20.19 ^
10 20.19 SyntaxError: Missing parentheses in call to 'print'. Did you mean print('IBM DataServer environment not set. Please refer to the README for more information on building the driver')?
10 20.19 ----------------------------------------
10 20.19 WARNING: Discarding https://files.pythonhosted.org/packages/a4/05/c503669163d20f55a9675e10f9804e4bc84689c031e1353fba3b747421de/ibm_db-0.2.0.tar.gz#sha256=e30bfa803c5ac7579d036f6899cc73d9d3f62efa7629a4d9c014a9baad5cf504 (from https://pypi.org/simple/ibm-db/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
10 20.19 ERROR: Could not find a version that satisfies the requirement ibm_db (from versions: 0.2.0, 0.2.6, 0.2.7, 0.2.9, 0.2.9.1, 0.3.0, 0.4.0, 0.4.1, 0.6.0, 0.7.0, 0.7.2, 0.7.2.1, 0.7.2.5, 0.8.0, 1.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.0.6, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.4.1, 2.0.5, 2.0.5.1, 2.0.6.1, 2.0.7, 2.0.8a0, 2.0.8, 2.0.9, 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.0.4)
10 20.19 ERROR: No matching distribution found for ibm_db
In one year of the project, this has never happened. Please we need your help.