search

blue-yonder / turbodbc

Turbodbc is a Python module to access relational databases via the Open Database Connectivity (ODBC) interface. The module complies with the Python Database API Specification 2.0.
http://turbodbc.readthedocs.io/en/latest
MIT License
622 stars 85 forks source link

Encountered error while trying to install package turbodbc #365

Open areebxfusiontech opened 2 years ago

areebxfusiontech commented 2 years ago

Docker build is failing upon installing turbodbc in dockerfile. This Dockerfile was successfully built a week before but now it's failing even though there is no change in the application and the infrastructure.

Dockerfile

# FROM python:3.9-slim

# FROM tiangolo/uvicorn-gunicorn-fastapi:python3.9-slim

FROM tiangolo/meinheld-gunicorn-flask:python3.9

# Tell Python to not generate .pyc
ENV PYTHONDONTWRITEBYTECODE 1

# Turn off buffering
ENV PYTHONUNBUFFERED 1

RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
    && apt-get -y install --no-install-recommends \
    build-essential \ 
    default-libmysqlclient-dev \
    libpython3-dev \
    libboost-all-dev \
    python3-wheel \
    python3-dev \
    software-properties-common \
    libsasl2-modules-gssapi-mit \
    libsasl2-modules \
    unixodbc \
    unixodbc-dev \
    odbc-postgresql \
    tdsodbc 

CMD gunicorn --bind :2090 --workers 4 --threads 8 main:app --log-level debug --timeout 0

RUN wget -O mysql-connector-odbc-8.0.12-linux-debian9-x86-64bit.tar.gz https://github.com/ubisoft/mobydq/blob/master/scripts/drivers/mysql-connector-odbc-8.0.12-linux-debian9-x86-64bit.tar.gz?raw=true && \
    tar -xvzf mysql-connector-odbc-8.0.12-linux-debian9-x86-64bit.tar.gz \
    && cp mysql-connector-odbc-8.0.12-linux-debian9-x86-64bit/lib/libmyodbc8* /usr/lib/x86_64-linux-gnu/odbc/ \
    && rm -R mysql-connector-odbc-8.0.12-linux-debian9-x86-64bit

# Install requirements using pip
ADD req.txt .

RUN \
    apt-get update --yes && \
    pip install -U pip==22.0.4 setuptools==45.2.0 wheel==0.37.1 && \
    apt-get install curl sudo git make musl-dev --yes && \
    pip install -r req.txt && \
    python -c "import pyarrow; pyarrow.create_library_symlinks()" && \
    CFLAGS="-D_GLIBCXX_USE_CXX11_ABI=0" pip install --no-cache-dir turbodbc==4.5.3 && \
    apt-get remove make gcc musl-dev libffi-dev --yes &&\
    apt-get clean autoclean && apt-get autoremove --yes && \
    rm -rf /var/lib/{apt,dpkg,cache,log}/

COPY ./odbcinst.ini /etc/

# Set working directory
WORKDIR /app

# Copy files
ADD ./API /app
ENV PORT=2090

EXPOSE 2090

Req.txt

boto3
botocore
cachetools==5.0.0
certifi==2021.10.8
charset-normalizer==2.0.12
fastapi==0.65.2
Flask==2.0.3
Flask-Cors==3.0.10
Flask-Ext==0.1
flask-restplus==0.13.0
Flask-Session==0.4.0
duckdb
google-api-core==2.7.2
google-auth==2.6.6
google-cloud-core==2.3.0
google-cloud-storage==2.3.0
google-crc32c==1.3.0
google-resumable-media==2.3.2
googleapis-common-protos==1.56.0
idna==3.3
jmespath==1.0.0
numpy==1.21.6
pandas==1.3.5
pydantic==1.8.2
protobuf==3.20.1
pyparsing==2.4.7
pytest==6.2.4
psycopg2-binary==2.9.1
pytest-cov==2.12.1
python-dotenv==0.17.1
python-jenkins==1.7.0
python-multipart==0.0.5
PyYAML==5.4.1
pyasn1==0.4.8
pyasn1-modules==0.2.8
pymongo
python-dateutil==2.8.2
pytz==2022.1
requests==2.27.1
rsa==4.8
s3path
s3transfer==0.5.2
s3fs==2022.3.0
six==1.16.0
urllib3==1.26.9
vaex
vaex-arrow
typing-extensions==3.10.0.0
uvicorn==0.14.0
uvloop==0.14.0
watchgod==0.7
websockets==9.1
wrapt==1.12.1
Werkzeug
zipp==3.4.1
pyarrow>=7.0.0
pyodbc==4.0.34

Error:

Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Building wheels for collected packages: turbodbc Building wheel for turbodbc (setup.py): started Building wheel for turbodbc (setup.py): still running... Building wheel for turbodbc (setup.py): still running... Building wheel for turbodbc (setup.py): still running... Building wheel for turbodbc (setup.py): still running... Building wheel for turbodbc (setup.py): finished with status 'error' [91m error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [1248 lines of output] /usr/local/lib/python3.9/site-packages/setuptools/dist.py:724: UserWarning: Module pyarrow was already imported from /usr/local/lib/python3.9/site-packages/pyarrow/init.py, but /tmp/pip-install-64lv0rpv/turbodbc_99cb81698f1444109a3d8faae1e53b9b/.eggs/pyarrow-9.0.0-py3.9-linux-x86_64.egg is being added to sys.path pkg_resources.working_set.add(dist, replace=True)

Full Error: https://gist.github.com/areebxfusiontech/14c5be6a4900bee96ac65dea9631d480#file-gistfile1-txt-L1191

david-engelmann commented 2 years ago

It might be related to the versions of numpy and pandas. Try adjusting the following lines in the requirements.txt

...
numpy==1.21.6
pandas==1.3.5
...

From == to >= ie.

...
numpy>=1.21.6
pandas>=1.3.5
...