mkleehammer / pyodbc

Python ODBC bridge
https://github.com/mkleehammer/pyodbc/wiki
MIT No Attribution
2.89k stars 562 forks source link

Fix for #1082 libraries in Linux wheels #1084

Closed keitherskine closed 1 year ago

keitherskine commented 2 years ago

It appears that the libodbc.so and libltdl.so libraries are added to the linux build wheels by the "repair" step in Github Actions. This step adds any libraries it deems necessary to fulfil the "libraries" attribute in setup.py and also renames the linux build wheel files from "linux" to "manylinux". As far as I can tell, that last rename is not strictly necessary, and can't be separated out, so I'm disabling that entire repair step.

Also, a few other notes:

Please do try out these new build wheels when you get a chance.

v-chojas commented 2 years ago

macOS wheels include the libodbc.2.dylib and libltdl.7.dylib libraries, which don't appear to be problematic

I don't think that should be done either. While Mac doesn't have unixODBC by default, someone who has it (e.g. after installing msodbcsql) will encounter much confusion when pyodbc does something different than when the same connection is made in isql, and some ODBC drivers that call back into the DM will attempt to use the system one, which might not match. Suffice to say that multiple DM libs have caused a lot of problems in the past.

gordthompson commented 2 years ago

I just checked a couple of the Linux wheels in here

https://github.com/mkleehammer/pyodbc/suites/7440010116/artifacts/304197945

and they look good to me. I do understand why some people would "like" to avoid installing unixODBC themselves, but they are simply unaware of the potential problems. Whatever the outcome—and I'm +1 on this approach—we'll need to update the "Installing on Linux" parts of the wiki, so we should remember to provide a brief explanation of why it's the way it is.

As for 32-bit Windows wheels, I would have been very keen on them a few years ago when I was doing a fair bit of MS Access stuff and Office installs were still mostly 32-bit. However, neither of those things is true now, and 32-bit wheels are still available at

https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyodbc

so I would be inclined to leave them out for the time being.

keitherskine commented 1 year ago

As mentioned over on issue 1082, my thoughts on this PR after all the discussion are as follows:

I agree we should spruce up the Wiki with better information about installing pyodbc for all OS's, but particularly for Linux and its many flavours. We should update the README as well. As ever, thoughts welcome.

gordthompson commented 1 year ago

One point in favour of providing Linux wheels (without the bundled libraries, that is):

Building pyodbc from source requires build tools, and installing them can consume a non-trivial amount of disk space. Not the several gigabytes of Visual Studio goodness required to build pyodbc on Windows, but still …

Users of distros like Alpine Linux might be particularly vocal about this. On my Alpine Linux VM with

apk install python3 unixodbc
python3 -m ensurepip

df -h shows 179.6M used on /. After I do

apk add python3-dev g++ unixodbc-dev

the used space has increased to 393.3M. So installing just the stuff to build pyodbc more than doubles the used space.

gordthompson commented 1 year ago

Also, re: 32-bit Windows wheels - If they're easy enough for us to provide, then it's probably a good idea. There could very well be quite a few older oddball Windows ODBC drivers that are 32-bit only, so if somebody needs to use one then at least we could save them the trouble of having to build pyodbc too.

keitherskine commented 1 year ago

Good to hear everybody's thoughts on this. They are greatly appreciated. Further to that, I've updated this PR with what I hope is the general prevailing opinions:

Please take a look at the artifacts generated by this latest commit and give your thoughts on them:

https://github.com/mkleehammer/pyodbc/actions/runs/2757580203

keitherskine commented 1 year ago

Many thanks for checking the latest builds against various platforms @gordthompson . I have checked the macOS Intel builds and I'm just waiting for a friend to confirm the M1 builds.

keitherskine commented 1 year ago

I haven't heard back from my friend about M1's but I'm going to merge this PR anyway. If we need to address this further, we can do so before the next release.