bbalasub1 / glmnet_python

GNU General Public License v3.0
199 stars 94 forks source link

"No suitable image found" on Mac OS X #17

Closed lmackey closed 5 years ago

lmackey commented 7 years ago

I'm encountering the same issue that @PeterJacob described in #13 . I'm running OS X version 10.11.6 (15G1217), and I've tried installing glmnet python via "pip install glmnet_py" and via the source installation instructions. In both cases the installation succeeds, and I'm able to "import glmnet_py" and "import glmnet_python," but when I run "glmnet(x = X.values.astype('float64'), y = y.values, family = 'cox')" in Python 3.6, I get the error

`--------------------------------------------------------------------------- OSError Traceback (most recent call last) in () 1 from glmnet_python import glmnet ----> 2 glmnet(x = X.values.astype('float64'), y = y.values)

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site-packages/glmnet_python/glmnet.py in glmnet(x, y, family, **options) 448 fit = elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm, 449 lempty, nvars, jd, vp, cl, ne, nx, nlam, flmin, ulam, --> 450 thresh, isd, intr, maxit, family) 451 elif (family == 'binomial') or (family == 'multinomial'): 452 # call lognet

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site-packages/glmnet_python/elnet.py in elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm, lempty, nvars, jd, vp, cl, ne, nx, nlam, flmin, ulam, thresh, isd, intr, maxit, family) 15 16 # load shared fortran library ---> 17 glmlib = loadGlmLib() 18 19 # pre-process data

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site-packages/glmnet_python/loadGlmLib.py in loadGlmLib() 22 def loadGlmLib(): 23 if os.name == 'posix': ---> 24 glmlib = ctypes.cdll.LoadLibrary(glmnet_so) 25 return(glmlib) 26 elif os.name == 'nt':

/Users/lmackey/anaconda/envs/python3/lib/python3.6/ctypes/init.py in LoadLibrary(self, name) 424 425 def LoadLibrary(self, name): --> 426 return self._dlltype(name) 427 428 cdll = LibraryLoader(CDLL)

/Users/lmackey/anaconda/envs/python3/lib/python3.6/ctypes/init.py in init(self, name, mode, handle, use_errno, use_last_error) 346 347 if handle is None: --> 348 self._handle = _dlopen(self._name, mode) 349 else: 350 self._handle = handle

OSError: dlopen(/Users/lmackey/anaconda/envs/python3/lib/python3.6/site-packages/glmnet_python/GLMnet.so, 6): no suitable image found. Did find: /Users/lmackey/anaconda/envs/python3/lib/python3.6/site-packages/glmnet_python/GLMnet.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00 `

I've tried the libgfortran.3.dylib fix that @PeterJacob suggested, but my traceback is unchanged. @PeterJacob suggested that he may have had an incorrect GLMnet.so file; is a corrected GLMnet.so available for Mac OS X?

Thanks!

bbalasub1 commented 7 years ago

Lester,

Unfortunately, I only have access to linux machines now and this was only tested on linux machines extensively. Others have had mixed luck with other OS.

It would be great to fix this problem for OSX. Any chance you might have some time to help?

On Sun, Jul 16, 2017 at 3:57 PM, Lester Mackey notifications@github.com wrote:

I'm encountering the same issue that @PeterJacob https://github.com/peterjacob described in #13 https://github.com/bbalasub1/glmnet_python/issues/13 . I'm running OS X version 10.11.6 (15G1217), and I've tried installing glmnet python via "pip install glmnet_py" and via the source installation instructions. In both cases the installation succeeds, and I'm able to "import glmnet_py" and "import glmnet_python," but when I run "glmnet(x = X.values.astype('float64'), y = y.values, family = 'cox')" in Python 3.6, I get the error

`-----------------------------------------------------------

OSError Traceback (most recent call last) in () 1 from glmnet_python import glmnet ----> 2 glmnet(x = X.values.astype('float64'), y = y.values)

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/glmnet.py in glmnet(x, y, family, **options) 448 fit = elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm, 449 lempty, nvars, jd, vp, cl, ne, nx, nlam, flmin, ulam, --> 450 thresh, isd, intr, maxit, family) 451 elif (family == 'binomial') or (family == 'multinomial'): 452 # call lognet

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/elnet.py in elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm, lempty, nvars, jd, vp, cl, ne, nx, nlam, flmin, ulam, thresh, isd, intr, maxit, family) 15 16 # load shared fortran library ---> 17 glmlib = loadGlmLib() 18 19 # pre-process data

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/loadGlmLib.py in loadGlmLib() 22 def loadGlmLib(): 23 if os.name == 'posix': ---> 24 glmlib = ctypes.cdll.LoadLibrary(glmnet_so) 25 return(glmlib) 26 elif os.name == 'nt':

/Users/lmackey/anaconda/envs/python3/lib/python3.6/ctypes/init.py in LoadLibrary(self, name) 424 425 def LoadLibrary(self, name): --> 426 return self._dlltype(name) 427 428 cdll = LibraryLoader(CDLL)

/Users/lmackey/anaconda/envs/python3/lib/python3.6/ctypes/init.py in init(self, name, mode, handle, use_errno, use_last_error) 346 347 if handle is None: --> 348 self._handle = _dlopen(self._name, mode) 349 else: 350 self._handle = handle

OSError: dlopen(/Users/lmackey/anaconda/envs/python3/lib/ python3.6/site-packages/glmnet_python/GLMnet.so, 6): no suitable image found. Did find: /Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/GLMnet.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00 `

I've tried the libgfortran.3.dylib fix that @PeterJacob https://github.com/peterjacob suggested, but my traceback is unchanged. @PeterJacob https://github.com/peterjacob suggested that he may have had an incorrect GLMnet.so file; is a corrected GLMnet.so available for Mac OS X?

Thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/17, or mute the thread https://github.com/notifications/unsubscribe-auth/AKpDE8kd9uupI-iNAUDht5UEAiHkvbA3ks5sOmtAgaJpZM4OZZY- .

lmackey commented 7 years ago

Sure, how can I help?

2017-07-16 18:29 GMT-04:00 bbalasub1 notifications@github.com:

Lester,

Unfortunately, I only have access to linux machines now and this was only tested on linux machines extensively. Others have had mixed luck with other OS.

It would be great to fix this problem for OSX. Any chance you might have some time to help?

On Sun, Jul 16, 2017 at 3:57 PM, Lester Mackey notifications@github.com wrote:

I'm encountering the same issue that @PeterJacob https://github.com/peterjacob described in #13 https://github.com/bbalasub1/glmnet_python/issues/13 . I'm running OS X version 10.11.6 (15G1217), and I've tried installing glmnet python via "pip install glmnet_py" and via the source installation instructions. In both cases the installation succeeds, and I'm able to "import glmnet_py" and "import glmnet_python," but when I run "glmnet(x = X.values.astype('float64'), y = y.values, family = 'cox')" in Python 3.6, I get the error

`-----------------------------------------------------------

OSError Traceback (most recent call last) in () 1 from glmnet_python import glmnet ----> 2 glmnet(x = X.values.astype('float64'), y = y.values)

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/glmnet.py in glmnet(x, y, family, **options) 448 fit = elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm, 449 lempty, nvars, jd, vp, cl, ne, nx, nlam, flmin, ulam, --> 450 thresh, isd, intr, maxit, family) 451 elif (family == 'binomial') or (family == 'multinomial'): 452 # call lognet

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/elnet.py in elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm, lempty, nvars, jd, vp, cl, ne, nx, nlam, flmin, ulam, thresh, isd, intr, maxit, family) 15 16 # load shared fortran library ---> 17 glmlib = loadGlmLib() 18 19 # pre-process data

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/loadGlmLib.py in loadGlmLib() 22 def loadGlmLib(): 23 if os.name == 'posix': ---> 24 glmlib = ctypes.cdll.LoadLibrary(glmnet_so) 25 return(glmlib) 26 elif os.name == 'nt':

/Users/lmackey/anaconda/envs/python3/lib/python3.6/ctypes/init.py in LoadLibrary(self, name) 424 425 def LoadLibrary(self, name): --> 426 return self._dlltype(name) 427 428 cdll = LibraryLoader(CDLL)

/Users/lmackey/anaconda/envs/python3/lib/python3.6/ctypes/init.py in init(self, name, mode, handle, use_errno, use_last_error) 346 347 if handle is None: --> 348 self._handle = _dlopen(self._name, mode) 349 else: 350 self._handle = handle

OSError: dlopen(/Users/lmackey/anaconda/envs/python3/lib/ python3.6/site-packages/glmnet_python/GLMnet.so, 6): no suitable image found. Did find: /Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/GLMnet.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00 `

I've tried the libgfortran.3.dylib fix that @PeterJacob https://github.com/peterjacob suggested, but my traceback is unchanged. @PeterJacob https://github.com/peterjacob suggested that he may have had an incorrect GLMnet.so file; is a corrected GLMnet.so available for Mac OS X?

Thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/17, or mute the thread https://github.com/notifications/unsubscribe-auth/AKpDE8kd9uupI- iNAUDht5UEAiHkvbA3ks5sOmtAgaJpZM4OZZY- .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/17#issuecomment-315642594, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMWMu-_u6uHgSxfRvzAoWqrX6yWZx58ks5sOo6ugaJpZM4OZZY- .

bbalasub1 commented 7 years ago

If you debug this/troubleshoot, please let us know with solutions. Meanwhile, I will attempt to find a machine on the cloud to reproduce this problem.

On Sun, Jul 16, 2017 at 6:38 PM, Lester Mackey notifications@github.com wrote:

Sure, how can I help?

2017-07-16 18:29 GMT-04:00 bbalasub1 notifications@github.com:

Lester,

Unfortunately, I only have access to linux machines now and this was only tested on linux machines extensively. Others have had mixed luck with other OS.

It would be great to fix this problem for OSX. Any chance you might have some time to help?

On Sun, Jul 16, 2017 at 3:57 PM, Lester Mackey <notifications@github.com

wrote:

I'm encountering the same issue that @PeterJacob https://github.com/peterjacob described in #13 https://github.com/bbalasub1/glmnet_python/issues/13 . I'm running OS

X

version 10.11.6 (15G1217), and I've tried installing glmnet python via "pip install glmnet_py" and via the source installation instructions. In both cases the installation succeeds, and I'm able to "import glmnet_py" and "import glmnet_python," but when I run "glmnet(x = X.values.astype('float64'), y = y.values, family = 'cox')" in Python 3.6, I get the error

`-----------------------------------------------------------

OSError Traceback (most recent call last) in () 1 from glmnet_python import glmnet ----> 2 glmnet(x = X.values.astype('float64'), y = y.values)

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/glmnet.py in glmnet(x, y, family, **options) 448 fit = elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm, 449 lempty, nvars, jd, vp, cl, ne, nx, nlam, flmin, ulam, --> 450 thresh, isd, intr, maxit, family) 451 elif (family == 'binomial') or (family == 'multinomial'): 452 # call lognet

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/elnet.py in elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm, lempty, nvars, jd, vp, cl, ne, nx, nlam, flmin, ulam, thresh, isd, intr, maxit, family) 15 16 # load shared fortran library ---> 17 glmlib = loadGlmLib() 18 19 # pre-process data

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/loadGlmLib.py in loadGlmLib() 22 def loadGlmLib(): 23 if os.name == 'posix': ---> 24 glmlib = ctypes.cdll.LoadLibrary(glmnet_so) 25 return(glmlib) 26 elif os.name == 'nt':

/Users/lmackey/anaconda/envs/python3/lib/python3.6/ctypes/init.py in LoadLibrary(self, name) 424 425 def LoadLibrary(self, name): --> 426 return self._dlltype(name) 427 428 cdll = LibraryLoader(CDLL)

/Users/lmackey/anaconda/envs/python3/lib/python3.6/ctypes/init.py in init(self, name, mode, handle, use_errno, use_last_error) 346 347 if handle is None: --> 348 self._handle = _dlopen(self._name, mode) 349 else: 350 self._handle = handle

OSError: dlopen(/Users/lmackey/anaconda/envs/python3/lib/ python3.6/site-packages/glmnet_python/GLMnet.so, 6): no suitable image found. Did find: /Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/GLMnet.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00 `

I've tried the libgfortran.3.dylib fix that @PeterJacob https://github.com/peterjacob suggested, but my traceback is unchanged. @PeterJacob https://github.com/peterjacob suggested that he may have had an incorrect GLMnet.so file; is a corrected GLMnet.so available for Mac OS X?

Thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/17, or mute the thread https://github.com/notifications/unsubscribe-auth/AKpDE8kd9uupI- iNAUDht5UEAiHkvbA3ks5sOmtAgaJpZM4OZZY- .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/17# issuecomment-315642594, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMWMu-_ u6uHgSxfRvzAoWqrX6yWZx58ks5sOo6ugaJpZM4OZZY- .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/17#issuecomment-315643034, or mute the thread https://github.com/notifications/unsubscribe-auth/AKpDEwpA0FU3vDgtqSaDYjFb--ehnYrMks5sOpDTgaJpZM4OZZY- .

lmackey commented 7 years ago

OK, I don't have any thoughts about how to proceed at the moment, but if others have suggestions, I'd be happy to try them.

2017-07-16 18:40 GMT-04:00 bbalasub1 notifications@github.com:

If you debug this/troubleshoot, please let us know with solutions. Meanwhile, I will attempt to find a machine on the cloud to reproduce this problem.

On Sun, Jul 16, 2017 at 6:38 PM, Lester Mackey notifications@github.com

wrote:

Sure, how can I help?

2017-07-16 18:29 GMT-04:00 bbalasub1 notifications@github.com:

Lester,

Unfortunately, I only have access to linux machines now and this was only tested on linux machines extensively. Others have had mixed luck with other OS.

It would be great to fix this problem for OSX. Any chance you might have some time to help?

On Sun, Jul 16, 2017 at 3:57 PM, Lester Mackey < notifications@github.com

wrote:

I'm encountering the same issue that @PeterJacob https://github.com/peterjacob described in #13 https://github.com/bbalasub1/glmnet_python/issues/13 . I'm running OS

X

version 10.11.6 (15G1217), and I've tried installing glmnet python via "pip install glmnet_py" and via the source installation instructions. In both cases the installation succeeds, and I'm able to "import glmnet_py" and "import glmnet_python," but when I run "glmnet(x = X.values.astype('float64'), y = y.values, family = 'cox')" in Python 3.6, I get the error

`-----------------------------------------------------------

OSError Traceback (most recent call last) in () 1 from glmnet_python import glmnet ----> 2 glmnet(x = X.values.astype('float64'), y = y.values)

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/glmnet.py in glmnet(x, y, family, **options) 448 fit = elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm, 449 lempty, nvars, jd, vp, cl, ne, nx, nlam, flmin, ulam, --> 450 thresh, isd, intr, maxit, family) 451 elif (family == 'binomial') or (family == 'multinomial'): 452 # call lognet

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/elnet.py in elnet(x, is_sparse, irs, pcs, y, weights, offset, gtype, parm, lempty, nvars, jd, vp, cl, ne, nx, nlam, flmin, ulam, thresh, isd, intr, maxit, family) 15 16 # load shared fortran library ---> 17 glmlib = loadGlmLib() 18 19 # pre-process data

/Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/loadGlmLib.py in loadGlmLib() 22 def loadGlmLib(): 23 if os.name == 'posix': ---> 24 glmlib = ctypes.cdll.LoadLibrary(glmnet_so) 25 return(glmlib) 26 elif os.name == 'nt':

/Users/lmackey/anaconda/envs/python3/lib/python3.6/ctypes/init.py in LoadLibrary(self, name) 424 425 def LoadLibrary(self, name): --> 426 return self._dlltype(name) 427 428 cdll = LibraryLoader(CDLL)

/Users/lmackey/anaconda/envs/python3/lib/python3.6/ctypes/init.py in init(self, name, mode, handle, use_errno, use_last_error) 346 347 if handle is None: --> 348 self._handle = _dlopen(self._name, mode) 349 else: 350 self._handle = handle

OSError: dlopen(/Users/lmackey/anaconda/envs/python3/lib/ python3.6/site-packages/glmnet_python/GLMnet.so, 6): no suitable image found. Did find: /Users/lmackey/anaconda/envs/python3/lib/python3.6/site- packages/glmnet_python/GLMnet.so: unknown file type, first eight bytes: 0x7F 0x45 0x4C 0x46 0x02 0x01 0x01 0x00 `

I've tried the libgfortran.3.dylib fix that @PeterJacob https://github.com/peterjacob suggested, but my traceback is unchanged. @PeterJacob https://github.com/peterjacob suggested that he may have had an incorrect GLMnet.so file; is a corrected GLMnet.so available for Mac OS X?

Thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/17, or mute the thread https://github.com/notifications/unsubscribe-auth/AKpDE8kd9uupI- iNAUDht5UEAiHkvbA3ks5sOmtAgaJpZM4OZZY- .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/17# issuecomment-315642594, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMWMu-_ u6uHgSxfRvzAoWqrX6yWZx58ks5sOo6ugaJpZM4OZZY- .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/17# issuecomment-315643034, or mute the thread https://github.com/notifications/unsubscribe- auth/AKpDEwpA0FU3vDgtqSaDYjFb--ehnYrMks5sOpDTgaJpZM4OZZY-

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/17#issuecomment-315643128, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMWMj5dRRBtrXtp9KlIIJdbW6juuj4yks5sOpFGgaJpZM4OZZY- .

PeterJacob commented 7 years ago

Yes this looks very similar to the error I was getting. I'm sorry to see that just copying libgfortran.3.dylib to the correct/magic location doesn't work. I'm running OS-X 10.12.5.

lmackey commented 7 years ago

Dear Peter,

You mentioned that you also had to replace the GLMnet.so with your colleague's version. Would it be possible to share the version of the GLMnet.so file that worked for you?

On Tue, Jul 18, 2017 at 4:45 AM Peter Smit notifications@github.com wrote:

Yes this looks very similar to the error I was getting. I'm sorry to see that just copying libgfortran.3.dylib to the correct/magic location doesn't work. I'm running OS-X 10.12.5.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/17#issuecomment-315998944, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMWMkxMfzHfkJ2vaYDYaMolWY4xVnxRks5sPHCogaJpZM4OZZY- .

-- Sent from my phone

PeterJacob commented 6 years ago

Sorry, I can't find that one anymore. Would it be difficult to add some logging to the Fortran code? I can't write fortran, but printing locations where it expects to find libraries should be possible right?

mzhao94 commented 6 years ago

I'm having the same issue on Mac OS X. Are there any updates on a solution?

bbalasub1 commented 6 years ago

Sorry. None at this time.

On Tuesday, November 14, 2017, mzhao94 notifications@github.com wrote:

I'm having the same issue on Mac OS X. Are there any updates on a solution?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/bbalasub1/glmnet_python/issues/17#issuecomment-344433582, or mute the thread https://github.com/notifications/unsubscribe-auth/AKpDE1Eli9w5uZhrX6mwvZrBWPcfCEurks5s2iCjgaJpZM4OZZY- .

marnixkoops commented 6 years ago

I have fixed the problem by first placing the libgfortran.3.dylib file in the location suggested by PeterJacob. First I located the file with:

$ sudo find / -name libgfortran.3.dylib

Next move the file as suggested:

mkdir -p /usr/local/opt/gcc/lib/gcc/6/
cp /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libgfortran.3.dylib 
/usr/local/opt/gcc/lib/gcc/6/

After that I installed gfortran from here Then it still did not work. The final step which fixed the problem for me was replacing my GLMnet.so file with the one from a computer where the error was not happening. See the attached zip: GLMnet.so.zip

During this process I replaced the old file so unfortunately I can not check what exactly the difference is which may be causing this trouble.

@mzhao94

jcusick13 commented 5 years ago

Was running into the same issues as Lester above but @marnixkoops' solution (and updated GLMnet.so file) did the trick for me. Thanks!

jadianes commented 5 years ago

In my case @marnixkoops solution almost fixed, but I had to copy the file to a different location (I think it will show up on the error itself):

cp /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libgfortran.3.dylib /usr/local/gfortran/lib/
bbalasub1 commented 5 years ago

Looks like this problem is fixed now thanks to @marnixkoops . Please reopen if you are running into further issues.