search

blue-yonder / turbodbc

Turbodbc is a Python module to access relational databases via the Open Database Connectivity (ODBC) interface. The module complies with the Python Database API Specification 2.0.
http://turbodbc.readthedocs.io/en/latest
MIT License
623 stars 85 forks source link

turbodbc does not play well with poetry #397

Closed bgemmill closed 12 months ago

bgemmill commented 1 year ago

When configuring a python project with poetry, issuing the command poetry add turbodbc fails with this error:

Using version ^4.7.0 for turbodbc

Updating dependencies
Resolving dependencies... (0.1s)

Package operations: 1 install, 0 updates, 0 removals

  • Installing turbodbc (4.7.0): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke get_requires_for_build_wheel

  Traceback (most recent call last):
...much backtrace spam goes here...
  ModuleNotFoundError: No module named 'pybind11'

Note: This error originates from the build backend, and is likely not a problem with poetry but with turbodbc (4.7.0) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "turbodbc (==4.7.0)"'.

Poetry seems right that this has something to do with turbodbc, in that running pip wheel --no-cache-dir --use-pep517 "turbodbc (==4.7.0)" produces the same error.

A straight pip install turbodbc does appear to work, where a wheel is compiled locally and installed.

Is this a pep517 thing, a lack of prebuilt wheel for ubuntu 22.04, or something else?

I'm happy to provide any sort of help to make this go; it would be great to get this into poetry.

rxm7706 commented 12 months ago

cc @xhochy @cshaley

Hi @bgemmill Thats an issue I was running into too - I believe the PR that just got merged https://github.com/blue-yonder/turbodbc/pull/401 will address this - Version 4.8.0 when released should be installable via pip, poetry and eventually conda too.

rxm7706 commented 12 months ago

@bgemmill will you be able to test this - I am getting still getting an error with Poetry & TurbODBC 4.9.0 If you are getting an error too - We should open a new issues 

 creating build/temp.linux-x86_64-cpython-310/src/turbodbc/result_sets
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/column_description.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/column_description.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/connection.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/connection.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/cpp_odbc.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/cpp_odbc.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/environment.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/environment.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/error.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/error.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level1/api.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level1/api.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level1/level1.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level1/level1.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level1/unixodbc_backend.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level1/unixodbc_backend.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level1/unixodbc_backend_debug.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level1/unixodbc_backend_debug.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level2/api.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level2/api.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level2/handles.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level2/handles.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level2/input_string_buffer.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level2/input_string_buffer.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level2/input_u16string_buffer.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level2/input_u16string_buffer.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level2/level1_connector.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level2/level1_connector.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level2/level2.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level2/level2.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level2/string_buffer.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level2/string_buffer.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level2/u16string_buffer.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level2/u16string_buffer.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level3/level3.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level3/level3.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level3/raii_connection.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level3/raii_connection.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level3/raii_environment.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level3/raii_environment.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/level3/raii_statement.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/level3/raii_statement.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/make_environment.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/make_environment.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/multi_value_buffer.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/multi_value_buffer.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/cpp_odbc/statement.cpp -o build/temp.linux-x86_64-cpython-310/src/cpp_odbc/statement.o --std=c++17
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Iinclude/ -I/tmp/tmpok_bmpw7/.venv/lib/python3.10/site-packages/pybind11/include -I/tmp/tmpok_bmpw7/.venv/include -I/opt/hostedtoolcache/Python/3.10.13/x64/include/python3.10 -c src/turbodbc/buffer_size.cpp -o build/temp.linux-x86_64-cpython-310/src/turbodbc/buffer_size.o --std=c++17
  In file included from include/turbodbc/description.h:3,
                   from include/turbodbc/buffer_size.h:3,
                   from src/turbodbc/buffer_size.cpp:1:
  include/turbodbc/field.h:3:10: fatal error: boost/variant/variant.hpp: No such file or directory
      3 | #include <boost/variant/variant.hpp>
        |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~
  compilation terminated.
  error: command '/usr/bin/gcc' failed with exit code 1

  at /opt/pipx/venvs/poetry/lib/python3.10/site-packages/poetry/installation/chef.py:164 in _prepare
      160│ 
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│ 
      163│             if error is not None:
    → 164│                 raise error from None
      165│ 
      166│             return path
      167│ 
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with turbodbc (4.9.0) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "turbodbc (==4.9.0)"'
xhochy commented 12 months ago

No, this error only signals that you have not installed the native dependencies required for building turbodbc on your system.

rxm7706 commented 12 months ago

No, this error only signals that you have not installed the native dependencies required for building turbodbc on your system.

Ah makes sense -- do we have a list of these dependencies.. Can you point me that list.

rxm7706 commented 11 months ago

I think I found what I was looking for - mostly.. https://github.com/blue-yonder/turbodbc/issues/351#issuecomment-1230512576 https://turbodbc.readthedocs.io/en/latest/pages/contributing.html https://turbodbc.readthedocs.io/en/latest/pages/contributing.html

# Install poetry package requirements
COPY poetry.lock pyproject.toml ./
RUN poetry install

# Hacky install turbodbc
RUN poetry run python -c "import pyarrow; pyarrow.create_library_symlinks()"
RUN CFLAGS="-D_GLIBCXX_USE_CXX11_ABI=0" poetry run pip install --no-cache-dir turbodbc
bgemmill commented 11 months ago

Dependencies are here

But if you have a local install of boost, it gets tricky with poetry and you may run into this https://github.com/python-poetry/poetry/issues/8727