search

lebedov / scikit-cuda

Python interface to GPU-powered libraries
http://scikit-cuda.readthedocs.org/
Other
984 stars 178 forks source link

OSX Support #29

Closed c0g closed 9 years ago

c0g commented 11 years ago

I am trying to run scikits.cuda on my OS10.8.3 apple laptop. I modified utils.py to look for libdl.dylib rather than libdl.so, and used homebrew to install gobjdump (I believe) help find the cublas version. I also tried changing the objdump invocation to gobjdump/llvm-objdump.

I'm now stuck on line 227 in utils.py, above which it says: 

XXX This approach to obtaining the CUBLAS version number may break Windows/MacOSX compatibility XXX

This appears to have happened. I'll probably keep plugging away at this throughout the day, but any chance of help?

lebedov commented 11 years ago

As I don't have access to any Apple hardware, I unfortunately can't provide much help in this case. If you do discover a successful method to extract a dylib soname, feel free to submit a patch.

c0g commented 11 years ago

I understand. I have found this which appears to say the command on OSX is 

otool -D

but I don't seem to get anything useful out of that (or gobjdump either). Is the below command useful? (I don't have access to a linux machine with CUDA so I don't know what the expected output of objdump is).

~ $ otool -L /usr/local/cuda/lib/libcublas.dylib
/usr/local/cuda/lib/libcublas.dylib:
@rpath/libcublas.dylib (compatibility version 1.1.0, current version 5.0.0)
@rpath/CUDA.framework/Versions/A/CUDA (compatibility version 1.1.0, current version 5.0.35)
@rpath/libcudart.dylib (compatibility version 1.1.0, current version 5.0.0)
/usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.9.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0)
lebedov commented 11 years ago

Running objdump -x path_to_lib/libcublas.so|grep SONAME results in something like this:

 SONAME                  libcublas.so.5.0

The output you sent seems like it might be useful. What do you get if you use the -D option with otool?

c0g commented 11 years ago

With -D I get:

~ $ otool -D /usr/local/cuda/lib/libcublas.dylib
/usr/local/cuda/lib/libcublas.dylib: 
@rpath/libcublas.dylib

I also ran all of the flags, available here: here. I grepped for SONAME but cannot see it. I skipped over the full disassembly in that file, the full 6meg version is here.

c0g commented 11 years ago 
p = subprocess.Popen(['otool','-L',filename],stdout=subprocess.PIPE)
out = p.communicate()[0]
value = re.search('^.*\s[0-9]',out.split(os.linesep)[1]).group(0)[-1]

Returns 5 when I have Cuda 5 installed, and 4 when I have Cuda 4. Is this the behaviour that we need? I'm having a look at integrating this with utils but it's going slowly :p

edit: I've forked to https://github.com/c0g/scikits.cuda.git. I have a version of utils.py there- it's a hack though, it just extract the 5 and puts it into libcublas.x.0 This still doesn't work, failing with a segfault if I try to run a BLAS function - here's the Mac's complaint: https://gist.github.com/c0g/5514690. This was caused by

result = cublas.cublasIsamax(self.cublas_handle, x_gpu.size, x_gpu.gpudata, 1)

in test_cublas.py. All of the other tests fail, mostly by segfaults except in the case of test_special.py which complains pycuda/pycuda-complex doesn't exist).

mforbes commented 11 years ago

I am not certain, but I think that the problems are cause by the use of ctypes.c_int() for the handle. I get crashes on my Macbook Pro with the following code:

import ctypes
_libcublas = ctypes.cdll.LoadLibrary('libcublas.dylib')
handle = ctypes.c_int()
status = _libcublas.cublasCreate_v2(ctypes.byref(handle))
_libcublas.cublasDestroy_v2(handle)

The following works however:

import ctypes
_libcublas = ctypes.cdll.LoadLibrary('libcublas.dylib')
handle = ctypes.c_void_p()
status = _libcublas.cublasCreate_v2(ctypes.byref(handle))
_libcublas.cublasDestroy_v2(handle)

Why are you using ints rather than pointers? Will try changing and then making a PR.

mforbes commented 11 years ago

Okay. I added a Pull Request (see above) that fixes most of the issues I had on Mac OS X. (I have not installed CULA, so I did not check the tests that failed because it was not installed.) See the PR for details of what was wrong and how I fixed it.

Michael.

lebedov commented 11 years ago

Thanks for all of the feedback and suggestions; I'll respond to the various items you addressed inline.

cnwangfeng commented 10 years ago

Hi,all. I still met the same error in my Mac OSX10.9. SCIKIT.CUDA is 0.43 WFMacBook-Pro:gpu wangfeng$ python gICLEAN.4.py Traceback (most recent call last): File "gICLEAN.4.py", line 17, in import scikits.cuda.fft as fft File "/Library/Python/2.7/site-packages/scikits.cuda-0.043-py2.7.egg/scikits/cuda/fft.py", line 19, in import misc File "/Library/Python/2.7/site-packages/scikits.cuda-0.043-py2.7.egg/scikits/cuda/misc.py", line 22, in import cublas File "/Library/Python/2.7/site-packages/scikits.cuda-0.043-py2.7.egg/scikits/cuda/cublas.py", line 224, in utils.get_soname(utils.find_lib_path('cublas'))).group(1) + '000') File "/Library/Python/2.7/site-packages/scikits.cuda-0.043-py2.7.egg/scikits/cuda/utils.py", line 43, in get_soname raise RuntimeError('error executing objdump') RuntimeError: error executing objdump

Shall I install the gobjdump? mforbes, Do you have a revision of utils.py?? Thx

lebedov commented 10 years ago

I integrated the updates from mforbes' pull request. Feel free to reopen this issue if the problem persists.

cnwangfeng commented 10 years ago

Yes, The problem is still exist. I have to download the full source from mforbes’ clone.

Dr Feng Wang Computer Technology Application Key Lab of Yunnan Province Kunming University of Science and Technology 650500 E-mail: wangfeng@acm.org

在 2014年1月8日 星期三，下午10:51，Lev Givon 写道：

I integrated the updates from mforbes' pull request. Feel free to reopen this issue if the problem persists.

— Reply to this email directly or view it on GitHub (https://github.com/lebedov/scikits.cuda/issues/29#issuecomment-31838615).

lebedov commented 10 years ago

What do you observe when you run the following with the latest code in master installed?

import scikits.cuda.utils as utils
lib_path = utils.find_lib_path('cublas')
print lib_path
print scikits.cuda.utils.get_soname(lib_path)
cnwangfeng commented 10 years ago

Python 2.7.5 (default, Aug 25 2013, 00:04:04)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin Type "help", "copyright", "credits" or "license" for more information.

import scikits.cuda.utils as utils lib_path = utils.find_lib_path('cublas') print lib_path None print scikits.cuda.utils.get_soname(lib_path) Traceback (most recent call last): File "", line 1, in NameError: name 'scikits' is not defined

Dr Feng Wang Computer Technology Application Key Lab of Yunnan Province Kunming University of Science and Technology 650500 E-mail: wangfeng@acm.org

在 2014年1月8日 星期三，下午11:03，Lev Givon 写道：

What do you observe when you run the following with the latest code in master installed? import scikits.cuda.utils as utils lib_path = utils.find_lib_path('cublas') print lib_path print scikits.cuda.utils.get_soname(lib_path)

— Reply to this email directly or view it on GitHub (https://github.com/lebedov/scikits.cuda/issues/29#issuecomment-31840064).

cnwangfeng commented 10 years ago

Dear Lev, maybe this would help to find the bugs.

File "gClean.py", line 20, in
import scikits.cuda.fft as fft File "/Library/Python/2.7/site-packages/scikits.cuda-0.043-py2.7.egg/scikits/cuda/fft.py", line 19, in import misc File "/Library/Python/2.7/site-packages/scikits.cuda-0.043-py2.7.egg/scikits/cuda/misc.py", line 22, in import cublas File "/Library/Python/2.7/site-packages/scikits.cuda-0.043-py2.7.egg/scikits/cuda/cublas.py", line 229, in utils.get_soname(utils.find_lib_path('cublas'))).group(1) + '000') File "/Library/Python/2.7/site-packages/scikits.cuda-0.043-py2.7.egg/scikits/cuda/utils.py", line 50, in get_soname raise RuntimeError('error executing {0}'.format(cmds)) RuntimeError: error executing ['otool', '-L', None]

Dr Feng Wang Computer Technology Application Key Lab of Yunnan Province Kunming University of Science and Technology 650500 E-mail: wangfeng@acm.org

在 2014年1月8日 星期三，下午11:03，Lev Givon 写道：

What do you observe when you run the following with the latest code in master installed? import scikits.cuda.utils as utils lib_path = utils.find_lib_path('cublas') print lib_path print scikits.cuda.utils.get_soname(lib_path)

— Reply to this email directly or view it on GitHub (https://github.com/lebedov/scikits.cuda/issues/29#issuecomment-31840064).

lebedov commented 10 years ago

You may need to install the Xcode Command Line Tools from Apple to get the otool command (you don't need to install Xcode itself).

cnwangfeng commented 10 years ago

No. I have otool command .

WFMacBook-Pro:Downloads wangfeng$ otool
Usage: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/otool [-arch arch_type] [-fahlLDtdorSTMRIHGvVcXmqQj] [-mcpu=arg] ... -f print the fat headers -a print the archive header -h print the mach header -l print the load commands -L print shared libraries used -D print shared library id name -t print the text section (disassemble with -v) -p start dissassemble from routine name -s print contents of section -d print the data section -o print the Objective-C segment -r print the relocation entries -S print the table of contents of a library -T print the table of contents of a dynamic shared library -M print the module table of a dynamic shared library -R print the reference table of a dynamic shared library -I print the indirect symbol table -H print the two-level hints table -G print the data in code table -v print verbosely (symbolically) when possible -V print disassembled operands symbolically -c print argument strings of a core file -X print no leading addresses or headers -m don't use archive(member) syntax -B force Thumb disassembly (ARM objects only) -q use llvm's disassembler (the default) -Q use otool(1)'s disassembler -mcpu=arg use `arg' as the cpu for disassembly -j print opcode bytes (ARM64 objects only)

BTW: I can use the mforbes’s codes. If I install the clone codes of mforbes, it seems everything is OK. :)

Dr Feng Wang Computer Technology Application Key Lab of Yunnan Province Kunming University of Science and Technology 650500 E-mail: wangfeng@acm.org

在 2014年1月8日 星期三，下午11:23，Lev Givon 写道：

You may need to install the Xcode (https://developer.apple.com/xcode/index.php) Command Line Tools from Apple to get the otool command (you don't need to install Xcode itself).

— Reply to this email directly or view it on GitHub (https://github.com/lebedov/scikits.cuda/issues/29#issuecomment-31843190).

lebedov commented 9 years ago

Closing - please reopen if the issue is still a problem.

  • © Githubissues.
  • Githubissues is a development platform for aggregating issues.