search

microsoft / homebrew-mssql-release

Other
91 stars 32 forks source link

[Driver Manager]Can't open lib file not found (0) Apple M1 #53

Closed MrRobot245 closed 3 years ago

MrRobot245 commented 3 years ago

Installed ODBC and msodbcsql mssql-tools all through brew on native ARM64 M1 Mac, install proceeds fine but it can't seem to find libmsodbcsql.17.dylib

I've tried symbolic linking to the /etc/ home-brew path like in the documents, and also changing the path inside the ini file to exactly the libmsodbcsql.17.dylib file. Chmod 777 the file and I still get this error:

[unixODBC][Driver Manager]Can't open lib '/opt/homebrew/lib/libmsodbcsql.17.dylib' : file not found (0) (SQLDriverConnect)"

/opt/homebrew/lib/ being the new Brew folder for M1 Macs

Edit: /opt/homebrew/lib/libmsodbcsql.17.dylib -> ../Cellar/msodbcsql17/17.6.1.1/lib/libmsodbcsql.17.dylib

v-chojas commented 3 years ago

No support for ARM64 at the moment.

MrRobot245 commented 3 years ago

That's what I figured, but it didn't get me an arch error so I thought it was something I could fix. Would this work under Rosetta2?

v-chojas commented 3 years ago

If your application and its dependencies are also consistently x86_64, I don't see why not.

LukeGahan commented 3 years ago

Running into the same issue report by @MrRobot245 and have been unable to resolve. Happy to provide more information if needed, understand support is only for x86_64 but would be good to know if there is a workaround to make use of Rosetta2

MrRobot245 commented 3 years ago

Mine did work with Rosetta2,

Kill the native brew version, duplicate your terminal all and check the Rosetta2 box and reinstall brew/MSSQL

Until there's a arm64 version that's what I'll be doing.

LukeGahan commented 3 years ago

Thanks @MrRobot245, those instructions worked for me, appreciate the help.

nico525 commented 3 years ago

Is there any timeline on arm? I do not want to remove and reinstall my full software stack just because of this ODBC driver FreeTDS works so far, but I am deploying to a Unix environment and would like to develop more closely to that environment

fonnesbeck commented 3 years ago

Is there any hope for a M1-compatible update, or is this not happening?

David-Engel commented 3 years ago

@nico525 @fonnesbeck M1 ARM support is being worked on. Barring any blocking issues, it is planned be in our next regular release at the end of July.

fonnesbeck commented 3 years ago

Thanks for the update.

wizardion commented 3 years ago

@David-Engel is there any news about the new version of this driver for M1 ARM? July is coming soon...

vwarchu commented 3 years ago

@wizardion - I'm filling in for David at the moment and can say that the end-of-July release is on track to include M1 ARM support.

victoriawallace commented 3 years ago

Hi @vwarchu - any update on the release for M1 ARM support?

David-Engel commented 3 years ago

@victoriawallace End of July == the last day of the month. If you want to get notifications of driver releases, you can subscribe to our blog at https://techcommunity.microsoft.com/t5/sql-server/bg-p/SQLServer/label-name/SQLServerDrivers

victoriawallace commented 3 years ago

Hi @David-Engel

I've tried reinstalling the ODBC Driver 17 for macOS following the release on Saturday -

$ odbcinst -j                                                                          
unixODBC 2.3.9
DRIVERS............: /etc/odbcinst.ini
SYSTEM DATA SOURCES: /etc/odbc.ini
FILE DATA SOURCES..: /etc/ODBCDataSources
USER DATA SOURCES..: /Users/vikki.wallace/odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8
$ cat /etc/odbcinst.ini                                           
[ODBC Driver 13 for SQL Server]
Description=Microsoft ODBC Driver 13 for SQL Server
Driver=/usr/local/lib/libmsodbcsql.13.dylib
UsageCount=2

[ODBC Driver 17 for SQL Server]
Description=Microsoft ODBC Driver 17 for SQL Server
Driver=/opt/homebrew/lib/libmsodbcsql.17.dylib
UsageCount=5
$ ls -la /opt/homebrew/lib/libmsodbcsql.17.dylib  
lrwxr-xr-x  1 vikki.wallace  admin  56 31 Jul 16:54 /opt/homebrew/lib/libmsodbcsql.17.dylib -> ../Cellar/msodbcsql17/17.8.1.1/lib/libmsodbcsql.17.dylib

but I still can't connect - 

>>> pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE=qai;UID='+username+';PWD='+password)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/homebrew/lib/libmsodbcsql.17.dylib' : file not found (0) (SQLDriverConnect)")
>>> pyodbc.connect('DRIVER={/opt/homebrew/Cellar/msodbcsql17/17.8.1.1/lib/libmsodbcsql.17.dylib};SERVER='+server+';DATABASE=qai;UID='+username+';PWD='+password)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/homebrew/Cellar/msodbcsql17/17.8.1.1/lib/libmsodbcsql.17.dylib' : file not found (0) (SQLDriverConnect)")

Can you please advise?

Thanks

RomkaSk commented 3 years ago

Everything works! Thank you!

RomkaSk commented 3 years ago

@victoriawallace

Try to use connection string like this

driver = 'ODBC Driver 17 for SQL Server'

SQLALCHEMY_CONNECTION_STRING = 'mssql+pyodbc://{2}:{3}@{0}/{1}?driver={4}&Port=1433'.format(
        DB_SERVER, DB_NAME, DB_USER, DB_PASSWORD, driver)

and after put it in sqlalchemy.engine.create_engine()

David-Engel commented 3 years ago

@RomkaSk Thanks for the heads up about the release notes. I just edited those and the mac install page to note the added M1 support. It should be reflected after tomorrow morning. Glad it's working for you!

zeljkolazic commented 3 years ago

Still doesn't work for me on Air M1. With driver = 'ODBC Driver 17 for SQL Server', I'm still getting

Can't open lib '/usr/local/lib/libmsodbcsql.17.dylib' : file not found

any help?

v-chojas commented 3 years ago

/usr/local/lib is where the x86 one should be. Did you install from a native ARM64 brew or one running under Rosetta2 emulation?

zeljkolazic commented 3 years ago

I installed it from the native ARM64 brew. When installed from brew under Rosetta2, the path is ../Cellar/msodbcsql17/17.8.1.1lib/libmsodbcsql.17.dylib, and it works fine in that case. But the point is that I want to remove brew under Rosetta2 and keep just the native one

v-chojas commented 3 years ago

Please try to remove it from the Rosetta2 brew, then install from the native one. Also make sure you are using native isql when testing connection, i.e. when running which isql result should be in /opt/homebrew/ instead of /usr/local.

absantiago commented 3 years ago

@RomkaSk I'm using the driver as you said 'driver = 'ODBC Driver 17 for SQL Server' ' but I'm still getting the error below

Can't open lib '/opt/homebrew/lib/libmsodbcsql.17.dylib' : file not found

Any help ?

v-chojas commented 3 years ago

Are you using native arm64 isql?

absantiago commented 3 years ago

@v-chojas I think yes, how can I check this ?

PS : When I type isql --version I get : unixODBC 2.3.9

JayBaywatch commented 3 years ago

I am having similar problems with the arm64 release.

I was getiting: Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/homebrew/lib/libmsodbcsql.17.dylib' : file not found (0) (SQLDriverConnect)")

Then I noticed that /opt/homebrew/lib/libmsodbcsql.17.dylib is linked to /opt/homebrew/Cellar/msodbcsql17/17.8.1.1/lib/libmsodbcsql.17.dylib

so I updated odbcinst.ini `[ODBC Driver 17 for SQL Server] Description=Microsoft ODBC Driver 17 for SQL Server

Driver=/opt/homebrew/lib/libmsodbcsql.17.dylib

Driver=/opt/homebrew/Cellar/msodbcsql17/17.8.1.1/lib/libmsodbcsql.17.dylib UsageCount=1`

But that didn't help. Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/homebrew/Cellar/msodbcsql17/17.8.1.1/lib/libmsodbcsql.17.dylib' : file not found (0) (SQLDriverConnect)")

Has anyone actually gotten this to work on Apple arm64 silicon? I tried on 2 separate machines with the same result.

yitam commented 3 years ago

I've installed homebrew from scratch and then ODBC driver and mssql-tools

  1. /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
  2. brew tap microsoft/mssql-release https://github.com/Microsoft/homebrew-mssql-release
  3. brew update
  4. HOMEBREW_NO_ENV_FILTERING=1 ACCEPT_EULA=Y brew install msodbcsql17 mssql-tools

image

I have no problem running a simple SELECT @@version using isql

image

JayBaywatch commented 3 years ago

something in our environments are different. Are those the defaults for you? My defaults are in /etc/ while yous appear to be in /opt/homebrew/etc/

unixODBC 2.3.9 DRIVERS............: /etc/odbcinst.ini SYSTEM DATA SOURCES: /etc/odbc.ini FILE DATA SOURCES..: /etc/ODBCDataSources USER DATA SOURCES..: /Users/baywatch/.odbc.ini SQLULEN Size.......: 8 SQLLEN Size........: 8 SQLSETPOSIROW Size.: 8

I do have /opt/homebrew/lib/libmsodbcsql.17.dylib but I get errors that ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib /opt/homebrew/lib/libmsodbcsql.17.dylib : file not found (0) (SQLDriverConnect)")

fresh install, and just got the same behavior.

yitam commented 3 years ago

Did you install unixodbc using homebrew, @JayBaywatch ? My env was basically clean... it was only used for testing installation. Please make sure your homebrew was installed in the native mode not Rosetta2

JayBaywatch commented 3 years ago

the first atempt, I installed unixodbc and freetds from homebrew. subsequent attempts, I let homebrew pull the dependencies from the msodbc recipe.

JayBaywatch commented 3 years ago

/usr/bin/arch reports arm64

v-chojas commented 3 years ago

Please remove and try again the installation.

JayBaywatch commented 3 years ago

hm, that works for isql now. but still no dice from python. Although I am no longer getting a complaint about/opt/homebrew/lib/libmsodbcsql.17.dylib not being found.

This is progress, thank you.

yitam commented 3 years ago

Great, @JayBaywatch !

For the others, to quickly check if isql is native, run file /opt/homebrew/bin/isql The expected output is /opt/homebrew/bin/isql: Mach-O 64-bit executable arm64

JayBaywatch commented 3 years ago

even using a minimalist sample from python I get errors. server = 'DB-DEV01' username = 'user' password = 'p@assword' pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE=Master;UID='+username+';PWD='+password)

Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/homebrew/lib/libmsodbcsql.17.dylib' : file not found (0) (SQLDriverConnect)")

isql still works though. weird.

v-chojas commented 3 years ago

Does otool -L '/opt/homebrew/lib/libmsodbcsql.17.dylib' show any "file not found" errors? You can also use dtruss to determine which file exactly it is not found.

JayBaywatch commented 3 years ago

Thanks, I had tried otool and everything seemed fine.

otool -L /opt/homebrew/lib/libmsodbcsql.17.dylib /opt/homebrew/lib/libmsodbcsql.17.dylib: /opt/homebrew/opt/msodbcsql17/lib/libmsodbcsql.17.dylib (compatibility version 0.0.1, current version 8.1.1) /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1770.255.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1) /opt/homebrew/lib/libodbcinst.2.dylib (compatibility version 3.0.0, current version 3.0.0) /System/Library/Frameworks/Kerberos.framework/Versions/A/Kerberos (compatibility version 5.0.0, current version 6.0.0) /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 904.4.0)

sqlcmd and isql both work, so I guess I have to assume that this is a pyodbc issue at this point.

edit: unsure if those compatibility versions are normal or not.

v-chojas commented 3 years ago

You are using ARM64 Python and pyODBC? https://github.com/mkleehammer/pyodbc/issues <- you can ask there

JayBaywatch commented 3 years ago

hey, thanks for that. There is a suggestion over there to build from source. I can try that.

zeljkolazic commented 3 years ago

hey, thanks for that. There is a suggestion over there to build from source. I can try that.

For me, it didn't work when running Python via Rosseta emulation although I had drivers properly installed. Switching to python 3.9.5 which runs nativly on ARM solved the issue. So, you need to run both python and msssql drivers nativly in order to make it work.

v-chojas commented 3 years ago

If everything is consistently running in emulation (from application to driver) that should work too, and would be your only choice if you have an x86 app that you need to use with the ODBC driver.

However, everything running natively in ARM mode should also work. The key point to remember is to not mix the two architectures' binaries together.

JayBaywatch commented 3 years ago

Unfortunately, runnning the arm64 version of conda would mean I don’t get some packages that I require, so it’s kind of a deal breaker.

We dev on Ubuntu and just recently got the green light for BYOD - so I’m trying to to get our toolkit some degree of compatibility here. I have everything else working but the data access bits.

Pymssql does work but it’s more of a hassle to logic fork that for m1’s then just trying to figure out if this will work or not. That said, it sounds and looks like its still a little premature to be successful.

On Thu, Sep 16, 2021 at 7:00 PM Željko Lazić @.***> wrote:

hey, thanks for that. There is a suggestion over there to build from source. I can try that.

For me, it didn't work when running Python via Rosseta emulation although I had drivers properly installed. Switching to python 3.9.5 which runs nativly on ARM solved the issue. So, you need to run both python and msssql drivers nativly in order to make it work.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/microsoft/homebrew-mssql-release/issues/53#issuecomment-921318856, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKNOKBSNLLLYUGFZSHJ6YXLUCJZJBANCNFSM4WN433XA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

JayBaywatch commented 3 years ago

That seems reasonable. Due to time constraints, for now, I’ll just see if I can monkey patch pymssql in when running on an m1.

But thank you for that suggestion. I’ll try to get a new env spun up that is all Rosetta based when I get some time.

On Thu, Sep 16, 2021 at 7:06 PM v-chojas @.***> wrote:

If everything is consistently running in emulation (from application to driver) that should work too, and would be your only choice if you have an x86 app that you need to use with the ODBC driver.

However, everything running natively in ARM mode should also work. The key point to remember is to not mix the two architectures' binaries together.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/microsoft/homebrew-mssql-release/issues/53#issuecomment-921321376, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKNOKBRHOFYW5CPNMITM6DLUCJZ7BANCNFSM4WN433XA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

JayBaywatch commented 3 years ago

just wanted to circle back to say thanks and that worked.

In case it's ever helpful to anyone else the steps are below.

  1. Uninstall msodbc 17: brew remove msodbcsql17 mssql-tools
  2. Remove any symlinks to odbcinst.ini and odbc.ini
  3. Remove pyodbc, if applicable (pip uninstall pyodbc)
  4. Install a separate instance of homebrew as outlined below: https://medium.com/swlh/run-x86-terminal-apps-like-homebrew-on-your-new-m1-mac-73bdc9b0f343
  5. in a terminal running Rosetta (as described in step 4) do the following steps below.
basnijholt commented 1 year ago

Thanks a lot @JayBaywatch!

I have written up some instructions to get pyodbc to work on the M1 as well:

One can only use pyodbc when using Rosetta (x86 emulation). Follow the steps below.

Uninstall M1 versions of brew packages (if installed at all):

brew uninstall unixodbc msodbcsql17 mssql-tools freetds

Install x86 Homebrew alongside the ARM M1 homebrew:

arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

Then use x86 homebrew like arch -x86_64 /usr/local/bin/brew install or use the following alias (add to ~/.bash_profile)

# Relies on having installed x86 brew like:
# arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
alias x86brew="arch -x86_64 /usr/local/bin/brew"
alias brew="/opt/homebrew/bin/brew"  # M1 version, to avoid from using x86 version accidentally

Install the ODBC packages.

x86brew tap microsoft/mssql-release https://github.com/Microsoft/homebrew-mssql-release
x86brew update
HOMEBREW_ACCEPT_EULA=Y x86brew install msodbcsql17 mssql-tools

Create an x86 conda env with:

ENV_NAME="rosetta"
CONDA_SUBDIR=osx-64 conda create -n $ENV_NAME python
conda activate $ENV_NAME
conda config --env --set subdir osx-64

or if using micromamba:

ENV_NAME="rosetta"
CONDA_SUBDIR=osx-64 micromamba create -n $ENV_NAME python
micromamba activate $ENV_NAME
joelharkes commented 1 year ago

i also get this error when i install v18 of the drivers 🤷 

Can't open lib '/opt/homebrew/lib/libmsodbcsql.17.dylib'
v-makouz commented 1 year ago

@joelharkes Why is it trying to open v17 dylib if you installed v18? What's the command that produces that error?

joelharkes commented 1 year ago

Idk, maybe i installed it earlier and it can't find it. But I uninstalled it yet it still looks for it. How could I debug it further?

v-makouz commented 1 year ago

What are the exact commands you're using?