Closed mahesh-panchal closed 7 years ago
Hello Mahesh,
Sorry to learn you have such difficulties getting pacasus up and running. The problem you face is not with pacasus itself, but with the opencl + pyopencl installation. I checked with others more involved in this. It looks like you have installed OpenCL headers that do not match the OpenCL ICD loader (libOpenCL.so). Pyopencl now thinks it's building against an ICD loader that supports OpenCL 2 (the SVM functions are part of OpenCL 2), but libOpenCL.so then apparently doesn't provide those functions. You could try to upgrade your OpenCl library to support OpenCl 2.0. If that is not an option, you can force pyopencl to work only with OpenCL 1.2. To do that, download the source, create (or edit) a file called 'siteconf.py' containing (at least) the line:
CL_PRETEND_VERSION = "1.2"
and rebuild:
rm -Rf build pip install .
Hope this helps.
I agree that a docker (or bioconda package) would be the preferred. OpenCL / CUDA are somewhat complicated to configure in an automated way and I just did not yet have the time to set it up.
Thank you for the quick reply.
I uninstalled and reinstalled pyOpenCL and now I have a new error which I don't understand.
pip uninstall pyOpenCL
git clone --recursive http://git.tiker.net/trees/pyopencl.git
cd pyopencl
echo "CL_PRETEND_VERSION = \"1.2\"" >> siteconf.py
pip install . --global-option=build_ext --global-option="-I$TOOLS/OpenCL/AMDAPPSDK-3.0/include"
When I run Pacasus, this is the error I get:
python pacasus.py $PHOME/B2017008_Novel_Fungus/03_Assemblies/Pacasus_Test/nanopore.fasta -o $PHOME/B2017008_Novel_Fungus/03_Assemblies/Pacasus_Test/nanopore.pacasus_filtered.fasta --framework=opencl --device_type=CPU --platform_name=NVIDIA
INFO - Initializing application...
DEBUG - Initializing Score...
DEBUG - Initializing score finished.
DEBUG - Initializing DnaRnaScore...
DEBUG - Creating matrix with parameters:
match_score: 3,
mismatch_score: -4,
gap_score: -3.0,
other_score: -1,
any_score: 0
DEBUG - Initializing DnaRnaScore finished.
INFO - Application initialized.
INFO - Setting program...
DEBUG - Initializing aligner...
DEBUG - Initializing hitlist...
DEBUG - Initializing hitlist OK.
DEBUG - Setting SW...
DEBUG - Using OpenCL CPU implementation
DEBUG - Initializing SmithWaterman.
Traceback (most recent call last):
File "pacasus.py", line 12, in <module>
ppw.run()
File "/afs/pdc.kth.se/roots/srv/tegner/v1.8/cfs/klemming/nobackup/m/mahpa906/Tools/Pacasus_env2.7/Pacasus/pacasus/pacasusall.py", line 106, in run
self._set_program()
File "/afs/pdc.kth.se/roots/srv/tegner/v1.8/cfs/klemming/nobackup/m/mahpa906/Tools/Pacasus_env2.7/Pacasus/pacasus/pacasusall.py", line 82, in _set_program
self.program = Palindrome(self.logger, self.score, self.settings)
File "/afs/pdc.kth.se/roots/srv/tegner/v1.8/cfs/klemming/nobackup/m/mahpa906/Tools/Pacasus_env2.7/Pacasus/pypaswas/pyPaSWAS/Core/Programs.py", line 357, in __init__
Aligner.__init__(self, logger, score, settings)
File "/afs/pdc.kth.se/roots/srv/tegner/v1.8/cfs/klemming/nobackup/m/mahpa906/Tools/Pacasus_env2.7/Pacasus/pypaswas/pyPaSWAS/Core/Programs.py", line 53, in __init__
self.smith_waterman = SmithWatermanCPU(self.logger, self.score, settings)
File "/afs/pdc.kth.se/roots/srv/tegner/v1.8/cfs/klemming/nobackup/m/mahpa906/Tools/Pacasus_env2.7/Pacasus/pypaswas/pyPaSWAS/Core/SmithWatermanOcl.py", line 320, in __init__
SmithWatermanOcl.__init__(self, logger, score, settings)
File "/afs/pdc.kth.se/roots/srv/tegner/v1.8/cfs/klemming/nobackup/m/mahpa906/Tools/Pacasus_env2.7/Pacasus/pypaswas/pyPaSWAS/Core/SmithWatermanOcl.py", line 38, in __init__
self._set_platform(self.settings.platform_name)
File "/afs/pdc.kth.se/roots/srv/tegner/v1.8/cfs/klemming/nobackup/m/mahpa906/Tools/Pacasus_env2.7/Pacasus/pypaswas/pyPaSWAS/Core/SmithWatermanOcl.py", line 89, in _set_platform
for platform in cl.get_platforms():
File "/afs/pdc.kth.se/roots/srv/tegner/v1.8/cfs/klemming/nobackup/m/mahpa906/Tools/Pacasus_env2.7/lib/python2.7/site-packages/pyopencl/cffi_cl.py", line 676, in get_platforms
_handle_error(_lib.get_platforms(platforms.ptr, platforms.size))
File "/afs/pdc.kth.se/roots/srv/tegner/v1.8/cfs/klemming/nobackup/m/mahpa906/Tools/Pacasus_env2.7/lib/python2.7/site-packages/pyopencl/cffi_cl.py", line 649, in _handle_error
raise e
pyopencl.cffi_cl.LogicError: clGetPlatformIDs failed: <unknown error -1001>
Googling hasn't really helped me here. I checked what was in:
$ ls -a /etc/OpenCL/vendors/
. .. nvidia.icd
The contents of that file are this:
$ cat /etc/OpenCL/vendors/nvidia.icd
libnvidia-opencl.so.1
Any light you could shine on this would be appreciated. Regards, Mahesh.
You're mixing two systems. The OpenCL vendor file indicated you have an NVIDIA GPU. In that case you should use: --framework=cuda --device_type=GPU --platform_name=NVIDIA and install the NVIDIA cuda SDK: https://developer.nvidia.com/cuda-downloads
But you installed the AMD SDK. If you'd like to run Pacasus on a CPU, you also need to install the AMD OpenCL driver: http://support.amd.com/en-us/kb-articles/Pages/OpenCL2-Driver.aspx
Could you post your system configuration? Make/model CPU and GPU, etc?
I see you're using nanopore data. These reads can be very long. For the very long reads (>25kb) the OpenCL CPU is the only option because of the memory requirements of the Smith-Waterman alignment involved.
Sorry, I was also trying things out. I also ran with the Intel parameter with the same results.
I thought I understood from the AMD SDK that it would also install the Intel drivers, but I have no idea if it did. There was no error with the installation of that.
The head node of the cluster I'm testing on has this config:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Model name: Intel(R) Xeon(R) CPU E5-2623 v3 @ 3.00GHz
Stepping: 2
CPU MHz: 3300.000
BogoMIPS: 6005.69
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 10240K
NUMA node0 CPU(s): 0-3,8-11
NUMA node1 CPU(s): 4-7,12-15
and the node I hope to run it on in future has this config
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 168
On-line CPU(s) list: 0-167
Thread(s) per core: 2
Core(s) per socket: 14
Socket(s): 6
NUMA node(s): 6
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Model name: Intel(R) Xeon(R) CPU E7-4850 v3 @ 2.20GHz
Stepping: 4
CPU MHz: 1200.203
BogoMIPS: 4406.63
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 35840K
NUMA node0 CPU(s): 0-13,84-97
NUMA node1 CPU(s): 14-27,98-111
NUMA node2 CPU(s): 28-41,112-125
NUMA node3 CPU(s): 42-55,126-139
NUMA node4 CPU(s): 56-69,140-153
NUMA node5 CPU(s): 70-83,154-167
I do not have root permissions on these..
Does the error mean I am missing the Intel drivers? What file exactly should be there? Is it possible for me to install these by myself or do I need root privileges?
You can also use the AMD loader for intel CPUs. But the ICD needs to in /etc, however, this page shows you how to get the ICD installed without root access: https://wiki.tiker.net/OpenCLHowTo#Per-user_ICD_registry.3F
Thank you. I think it's working now. The test command is doing something.
I'm not sure which part got it working since I was a little confused about the instructions. This is what I did though.
wget http://registrationcenter-download.intel.com/akdlm/irc_nas/9019/opencl_runtime_16.1.1_x64_ubuntu_6.4.0.25.tgz
tar -xvf opencl_runtime_16.1.1_x64_ubuntu_6.4.0.25.tgz
cd opencl_runtime_16.1.1_x64_ubuntu_6.4.0.25
TGT_DIR=$PWD/intel-opencl-icd-16.1.1/lib
mkdir -p $TGT_DIR
rpm2cpio rpm/opencl-1.2-intel-cpu-6.4.0.25-1.x86_64.rpm | cpio -idmv
cp ./opt/intel/opencl-1.2-6.4.0.25/lib64/* $TGT_DIR
echo $TGT_DIR/libintelocl.so > intel.icd
export OPENCL_VENDOR_PATH=$TOOLS/OpenCL/opencl_runtime_16.1.1_x64_ubuntu_6.4.0.25
Since I wasn't sure about linking the libOpenCL.so I did this.
LD_LIBRARY_PATH="$TOOLS/OpenCL/AMDAPPSDK-3.0/lib/x86_64/sdk:$LD_LIBRARY_PATH"
pip uninstall pyOpenCL
pip install . --global-option=build_ext --global-option="-I$TOOLS/OpenCL/AMDAPPSDK-3.0/include"
Thank you for your help. Regards, Mahesh.
No problem! Glad to hear you have it up and running. Installing opencl is a pain on most systems :-(
Let me know when you need help with pacasus itself.
Hello, I wonder if you can help. I've spend a while struggling to get Pacasus installed. In the end here are the instructions I used. I installed OpenCL using the AMD installer
And then installed Pacasus with.
When I try to run Pacasus, I get this error.
Could you help me resolve the error please?
It would be great if this tool could be bundled into a BioConda package or Docker container for easier installation.
Regards, Mahesh.