Closed outlace closed 8 years ago
Hi, sorry for the delay in replying; I like to think I normally answer faster than that. So... building on Mac is a bit hit and miss unfortunately, it depends on a few factors, works for some people, less well for others. I think you should try using the linux method for building, and see what happens, ie:
git clone --recursive https://github.com/hughperkins/DeepCL.git
cd DeepCL
mkdir build
cd build
ccmake ..
# in ccmake:
# - press 'c'/configure
# - choose the options you want
# - press 'c' /configure again
# - press 'g' / generate, then `q` / quit
make -j 4 install
./deepcl_unittests
More details on pre-requisites and stuff at https://github.com/hughperkins/DeepCL/blob/master/doc/Build.md
Edit: by the way, please let me know how this goes, success/failure, and any logs and so on, if any issues.
Sorry for late reply, was out of town.
Tried installing. Seemed to install okay, but I tried ./deepcl_unittests and get this error
$ ./deepcl_unittests dyld: Library not loaded: libEasyCL.dylib Referenced from: /Users/brandon/DeepCL/build/./deepcl_unittests Reason: image not found Trace/BPT trap: 5
Although I've installed/built EasyCL
Hi outlace, Ok, I think you will need to set some environment variables, such as DYLD_LIBRARY_PATH, something like:
export DYLD_LIBRARY_PATH=../dist/lib
export LD_LIBRARY_PATH=../dist/lib
./deepcl_unittests
I think it worked? Does that one failed test matter?
[ OK ] testGpuOp.addscalarinplace (8 ms) [----------] 4 tests from testGpuOp (35 ms total)
[----------] 1 test from testjpeghelper [ RUN ] testjpeghelper.writeread [ OK ] testjpeghelper.writeread (13 ms) [----------] 1 test from testjpeghelper (13 ms total)
[----------] Global test environment tear-down [==========] 159 tests from 29 test cases ran. (67203 ms total) [ PASSED ] 158 tests. [ FAILED ] 1 test, listed below: [ FAILED ] testNorbLoader.load1000
1 FAILED TEST YOU HAVE 2 DISABLED TESTS
That one test doesnt matter. It means that the norb dataset is not present, which is fine. Looks like it works, you have succeeded :-)
Sweet! Thank you very much!!
Actually.. Now I'm trying to install the Python wrapper and I get this error:
Failed building wheel for DeepCL Failed to build DeepCL Installing collected packages: DeepCL Running setup.py install for DeepCL Complete output from command /Users/brandon/anaconda/bin/python3 -c "import setuptools, tokenize;file='/private/var/folders/zt/qjk01pyj4hn34zyhq9h50r1c0000gn/T/pip-build-k35n57y6/DeepCL/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/zt/qjk01pyj4hn34zyhq9h50r1c0000gn/T/pip-sgk6frfv-record/install-record.txt --single-version-externally-managed --compile: version: 8.0.0 running install running build running build_ext building 'PyDeepCL' extension gcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/brandon/anaconda/include -arch x86_64 -Imysrc -Imysrc/lua -I/Users/brandon/anaconda/include/python3.4m -c PyDeepCL.cxx -o build/temp.macosx-10.5-x86_64-3.4/PyDeepCL.o In file included from PyDeepCL.cxx:251: In file included from /Users/brandon/DeepCL/dist/include/deepcl/DeepCL.h:6: In file included from /Users/brandon/DeepCL/dist/include/easycl/EasyCL.h:31: In file included from /Users/brandon/DeepCL/dist/include/easycl/deviceinfo_helper.h:22: /Users/brandon/DeepCL/dist/include/easycl/mystdint.h:11:14: fatal error: 'cstdint' file not found
^
1 error generated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/Users/brandon/anaconda/bin/python3 -c "import setuptools, tokenize;file='/private/var/folders/zt/qjk01pyj4hn34zyhq9h50r1c0000gn/T/pip-build-k35n57y6/DeepCL/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/zt/qjk01pyj4hn34zyhq9h50r1c0000gn/T/pip-sgk6frfv-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/zt/qjk01pyj4hn34zyhq9h50r1c0000gn/T/pip-build-k35n57y6/DeepCL
Looking at the setup.py file, I think that the problem is that under Darwin clang isn't told to set the '-std=c++0x' option.
@GOFAI Ah, ok. Any thoughts on how to update the setup.py to include this option?
Ah, in this bit:
if osfamily == 'Windows':
compile_options.append('/EHsc')
elif osfamily == 'Linux':
compile_options.append('-std=c++0x')
compile_options.append('-g')
else:
pass
outface, dont suppose.. can you open a python prompt, and do:
import platform
print platform.uname()[0]
import platform print(platform.uname()[0]) Darwin
Don't have time to test at the moment, but I think it's probably as simple as:
if osfamily == 'Windows':
compile_options.append('/EHsc')
elif osfamily == 'Linux' or osfamily == 'Darwin':
compile_options.append('-std=c++0x')
compile_options.append('-g')
else:
pass
Ok. Fixed in cdb0438 outface, can you pull down the latest version, and retry?
The above didn't work for me, but the following did!
if osfamily == 'Windows':
compile_options.append('/EHsc')
elif osfamily in ['Linux']:
compile_options.append('-std=c++0x')
compile_options.append('-g')
elif osfamily == 'Darwin':
compile_options.append('-mmacosx-version-min=10.7')
compile_options.append('-stdlib=libc++')
compile_options.append('-g')
Thanks! Merged.
I'm getting a different error now $ python setup.py install ... ... running install_lib running build_ext building 'PyDeepCL' extension creating build creating build/temp.macosx-10.5-x86_64-3.5 gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/brandonbrown/anaconda/include -arch x86_64 -Imysrc -Imysrc/lua -I/Users/brandonbrown/anaconda/include/python3.5m -c PyDeepCL.cxx -o build/temp.macosx-10.5-x86_64-3.5/PyDeepCL.o -mmacosx-version-min=10.7 -stdlib=libc++ -g PyDeepCL.cxx:250:10: fatal error: 'CppRuntimeBoundary.h' file not found
^
1 error generated. error: command 'gcc' failed with exit status 1
is it possible that you forgot to run source dist/bin/activate.sh
?
Whoops, indeed I did. Thanks
On Wednesday, November 4, 2015, Matthew Smith notifications@github.com wrote:
is it possible that you forgot to run source dist/bin/activate.sh ?
— Reply to this email directly or view it on GitHub https://github.com/hughperkins/DeepCL/issues/32#issuecomment-153925310.
Now I have an error importing. Sorry guys...Thanks so much for your patience and help
Brandons-Air:DeepCL brandon$ python Python 3.5.0 |Continuum Analytics, Inc.| (default, Oct 20 2015, 14:39:26) [GCC 4.2.1 (Apple Inc. build 5577)] on darwin Type "help", "copyright", "credits" or "license" for more information.
import PyDeepCL Traceback (most recent call last): File "
", line 1, in ImportError: dlopen(/Users/brandonbrown/anaconda/lib/python3.5/site-packages/DeepCL-0.0.0-py3.5-macosx-10.5-x86_64.egg/PyDeepCL.cpython-35m-darwin.so, 2): Library not loaded: libclBLAS.2.dylib Referenced from: /Users/brandonbrown/anaconda/lib/python3.5/site-packages/DeepCL-0.0.0-py3.5-macosx-10.5-x86_64.egg/PyDeepCL.cpython-35m-darwin.so Reason: image not found
Hmmmm, somehow missing the libclBLAS.2.dylib library in the classpath. It looks like the PyDeepCL library references it, but it's not being found.
There seem to be two possibilities I can think of:
The first possibility is easier to check, so let's check that first. Plesae can you do:
echo $DYLD_LIBRARY_PATH
ls /Users/brandon/DeepCL/dist/lib
Brandons-Air:DeepCL brandon$ echo $DYLD_LIBRARY_PATH
Brandons-Air:DeepCL brandon$ ls /Users/brandon/DeepCL/dist/lib cmake libclBLAS.2.4.0.dylib libclew.1.0.0.dylib libDeepCL.dylib libclBLAS.2.dylib libclew.dylib libEasyCL.dylib libclBLAS.dylib libdeepcl_gtest.a
hmmm, can you try setting your DYLD_LIBRARY_PATH, maybe something like:
export DYLD_LIBRARY_PATH=/Users/brandon/DeepCL/dist/lib
... and then retry using python
For some reason, I needed to set the DYLD_FALLBACK_LIBRARY_PATH environment variable...same as above just
export DYLD_FALLBACK_LIBRARY_PATH=/Users/brandon/DeepCL/dist/lib
That did it! You guys are amazing, thank you so much
Brandons-Air:DeepCL brandonbrown$ python Python 3.5.0 |Continuum Analytics, Inc.| (default, Oct 20 2015, 14:39:26) [GCC 4.2.1 (Apple Inc. build 5577)] on darwin Type "help", "copyright", "credits" or "license" for more information.
import PyDeepCL cl = PyDeepCL.DeepCL() Using Apple , OpenCL platform: Apple Using OpenCL device: HD Graphics 5000 initializing clblas
Ah, needed the DYLD_FALLBACK_LIBRARY_PATH? Cool. Nice! (sorry about my occasional repeated postings by the way, my network is a bit iffy :-P )
Added DYLD_LIBRARY_PATH and DYLD_FALLBACK_LIBRARY_PATH in 911f53b. I think I can close this now, right?
total n00b here... [ FAILED ] testsinglebatch.imagesize5_filtersize3_batchsize2_10filters [ FAILED ] testNorbLoader.load1000 [ FAILED ] testCopyLocal.basic
I have tried your instructions above... I am failing 2 more unit tests... testCopyLocal.basic sounds important...
Yes, the other two are ok to fail. norbloader fails, because it needs the norb dataset, which you dont have. The singlebatch test is a bit stochastic, and fails occasionally (I should fix that really).
testcopylocal doesnt normally fail.... in fact I dont remember it ever failing... if all the other tests are passing, it probably doesnt matter really though. I think you can probably just go ahead and use the library. However, if you do want to dig into why, I think you need to provide:
clinfo
commandtestCopyLocal.basic
testThanks! I'm using OSX Mavericks on a Macbook Air (late 2013) with the intel HD 5000 card (opencl enabled)... There's a reason why i'm looking for opencl... as I have not exploited this feature that much.
I'm attaching clinfo here... not sure how to retrieve the output of testCopyLocal.basic (shame shame on me) clinfo.txt
Ok. Can you paste the full output of running ./deepcl_unittests tests=testCopyLocal.*
please?
On my laptop, this gives me:
DeepCL/build$ ./deepcl_unittests tests=testCopyLocal.*
args: ./deepcl_unittests --gtest_filter=testCopyLocal.*
Note: Google Test filter = testCopyLocal.*
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from testCopyLocal
[ RUN ] testCopyLocal.basic
Using NVIDIA Corporation , OpenCL platform: NVIDIA CUDA
Using OpenCL device: GeForce 940M
0 0 0 0
1 2 3 4
5 6 7 8
9 10 11 12
0 0 0 0
[ OK ] testCopyLocal.basic (809 ms)
[----------] 1 test from testCopyLocal (809 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (809 ms total)
[ PASSED ] 1 test.
Hi Hugh,
Thanks for your patience. I've ran the test and dumped the error message into the text file below. error_msg.txt
Dexter
Ok... not quite sure how to fix that to be honest. Works ok on my machine. Let's at least get rid of some of those opencl build warnings (or try to...), and those two failing tests, which should be done (or at least, started) in commit 022b6a3 , on master
branch
Perhaps you can try running the EasyCL tests first, since presumably it's fairly on something fairly basic (I guess....). Please can you try the following? :
git clone --recursive https://github.com/hughperkins/EasyCL.git
cd EasyCL
mkdir build
cd build
ccmake ..
# press 'c' for configure, set:
# BUILD_TESTS=ON,
# CMAKE_BUILD_TYPE=Debug
# press 'c' for configure
# press 'g' for generate
# => ccmake should exit. Then do:
make -j 4 install
cp ../test/*.cl .
./easycl_unittests
... and provide the results.
Hi! Sorry just came from work. I've ran this... is the warning after pressing g in ccmake relevant --> "Policy cmp0042 is not set: macosx_rpath is enabled by default ..." easycltests.txt
is the warning after pressing g in ccmake relevant --> "Policy cmp0042 is not set: macosx_rpath is enabled by default ..."
dont think so, because the problem is probably opencl-related. rpath is used to locate libraries, ie .so
s and .dylib
s. If those weren't being located the program wouldnt run at all, would crash immediately with a message like couldnt find libOpenCL.so
, something like that.
easycltests.txt
Hmmm, those all pass...
i think this library is usable... i have tried running test_deepcl.py... and it seems to be running, showing a test accuracy of 95.62%
Ok, cool :-)
thanks!
Hi Hugh,
I think we can compile a readme for OSX installation.
I've compiled my experience here, and lifted some of your instructions.
http://mynotebookis.blogspot.sg/2015/11/installing-deepcl-on-macbook-air-2013.html
Might it be worthwhile to put together a formula for homebrew-science
, or should that wait until the next official release?
@Dexdev08 Thats awesome! Thanks! :-)
@GOFAI That sounds cool. Maybe I will fire up Jenkins and compile the latest code, and we use that?
Ok. Builds working now. Created v8.1.2 release https://github.com/hughperkins/DeepCL/releases/tag/v8.1.2
I've made two Homebrew formulas. The first, head-only one worked fine with brew install -i --HEAD DeepCL
:
https://gist.github.com/GOFAI/6071778631ba3d7e8ebf
It would be better to use the tarball of the versioned release, but unfortunately it turns out that if one tries that the recursive submodules EasyCL and clMathLibraries don't get cloned:
https://gist.github.com/GOFAI/6b07ecdbace369ddd4ca
When I try to step through this one in interactive mode (brew install -i DeepCL
), I hit the following snags in cmake:
Setting build type to 'RelWithDebInfo'
CMake Error at
/usr/local/Cellar/cmake/3.3.2/share/cmake/Modules/ExternalProject.cmake:1871
(message):
No download info given for 'EasyCL-external' and its source directory:
/tmp/deepcl20151120-8228-1sxnmek/DeepCL-8.1.2/EasyCL
is not an existing non-empty directory. Please specify one of:
* SOURCE_DIR with an existing non-empty directory
* URL
* GIT_REPOSITORY
* HG_REPOSITORY
* CVS_REPOSITORY and CVS_MODULE
* SVN_REVISION
* DOWNLOAD_COMMAND
Call Stack (most recent call first):
/usr/local/Cellar/cmake/3.3.2/share/cmake/Modules/ExternalProject.cmake:2356
(_ep_add_download_command)
cmake/build_EasyCL.cmake:3 (ExternalProject_Add)
CMakeLists.txt:66 (INCLUDE)
CMake Error at
/usr/local/Cellar/cmake/3.3.2/share/cmake/Modules/ExternalProject.cmake:1871
(message):
No download info given for 'clBLAS-external' and its source directory:
/tmp/deepcl20151120-8228-1sxnmek/DeepCL-8.1.2/clMathLibraries/clBLAS/src
is not an existing non-empty directory. Please specify one of:
* SOURCE_DIR with an existing non-empty directory
* URL
* GIT_REPOSITORY
* HG_REPOSITORY
* CVS_REPOSITORY and CVS_MODULE
* SVN_REVISION
* DOWNLOAD_COMMAND
Call Stack (most recent call first):
/usr/local/Cellar/cmake/3.3.2/share/cmake/Modules/ExternalProject.cmake:2356
(_ep_add_download_command)
cmake/build_clBLAS.cmake:9 (ExternalProject_Add)
CMakeLists.txt:70 (INCLUDE)
Nice! :-)
It would be better to use the tarball of the versioned release, but unfortunately it turns out that if one tries that the recursive submodules EasyCL and clMathLibraries don't get cloned
Hmmmm...ok... well... couple of ideas, not sure how doable these are?
git clone http://some/url -b v8.1.2
?Some version of the second method is canonical for brew formulae. It would be nice to be able to build against other brew formulae such as clBLAS
, but if need be there is apparently a way to pull in the subcomponents as versioned tar files. I just need to untar them into the right folders before running ccmake
, correct?
Yes, you will need to untar as follows:
So I've got clBLAS, EasyCL, and clew staging correctly, but I'm not sure what arguments to pass to cmake, and using homebrew's default *std_cmake_args
results in the following output:
JOHNNIAC:~ Walrus$ brew install DeepCL
==> Downloading https://github.com/hughperkins/DeepCL/archive/v8.1.2.tar.gz
Already downloaded: /Library/Caches/Homebrew/deepcl-8.1.2.tar.gz
==> Downloading https://github.com/hughperkins/EasyCL/archive/b9023cdf25c10524ad
Already downloaded: /Library/Caches/Homebrew/deepcl--EasyCL-2.zip
==> Downloading https://github.com/hughperkins/clBLAS/archive/bdaf1f9ef3d97168fd
Already downloaded: /Library/Caches/Homebrew/deepcl--clBLAS-171.zip
==> Downloading https://github.com/hughperkins/clew/archive/1d2752564b6af98d2412
Already downloaded: /Library/Caches/Homebrew/deepcl--clew-02.zip
==> mkdir build
==> cd build
==> cmake .. -DCMAKE_C_FLAGS_RELEASE= -DCMAKE_CXX_FLAGS_RELEASE= -DCMAKE_INSTALL
Last 15 lines from /Users/Walrus/Library/Logs/Homebrew/deepcl/03.cmake:
2015-11-21 12:13:36 -0800
cmake
..
-DCMAKE_C_FLAGS_RELEASE=
-DCMAKE_CXX_FLAGS_RELEASE=
-DCMAKE_INSTALL_PREFIX=/usr/local/Cellar/deepcl/8.1.2
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_FIND_FRAMEWORK=LAST
-DCMAKE_VERBOSE_MAKEFILE=ON
-Wno-dev
CMake Error: The source directory "/tmp/deepcl20151121-14817-1dqypbk" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
READ THIS: https://git.io/brew-troubleshooting
Sorted out the problem--builds successfully and ./deepcl_unittests passes 158 of 159 tests. I just need to clean it up to meet homebrew's submission standards, including some sort of nominal test.
One issue is that using deepcl_unittests as the required test will run afoul of Brew Test Bot, because the single failed test causes the whole thing to count as a failure:
JOHNNIAC:bin Walrus$ brew test DeepCL
Testing deepcl
==> /usr/local/Cellar/deepcl/8.1.2/bin/deepcl_unittests
Last 15 lines from /Users/Walrus/Library/Logs/Homebrew/deepcl/01.deepcl_unittests:
[----------] 1 test from testjpeghelper
[ RUN ] testjpeghelper.writeread
[ OK ] testjpeghelper.writeread (1 ms)
[----------] 1 test from testjpeghelper (1 ms total)
[----------] Global test environment tear-down
[==========] 159 tests from 30 test cases ran. (59058 ms total)
[ PASSED ] 158 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] DATA_testNorbLoader.load1000
1 FAILED TEST
YOU HAVE 2 DISABLED TESTS
Error: deepcl: failed
Oh.... seems I havent quite excluded the DATA_* tests (of which there are one). I'll fix that.
PS Nice work Edward :-)
How do you get this to work on Mac OSX?