Open warrenmcg opened 9 years ago
Hi warrencmg, I've experienced the same problems trying to use octave and therefore I'm following your steps (thank you so much for that! :D ). However, lapack is giving me a hard time and I cannot see what is missing in my make.inc file; every time it ends up in a loop on eig testing steps or the recipe for an eig target fails. So was your make.inc like that?
FORTRAN = gfortran OPTS = -O2 -frecursive -fdefault-integer-8 DRVOPTS = $(OPTS) NOOPT = -O0 -frecursive -fPIC LOADER = gfortran LOADOPTS =
thank you!
try: ... OPTS = -O2 -fdefault-integer-8 -fPIC ... NOOPT = -O0 -fdefault-integer-8 -fPIC
This will fix the problem I think.
Also while compiling the lapack module, please do the following:
cd BLAS/SRC`
make -j `grep ^processor /proc/cpuinfo | wc -l`
cd ../..
make -j `grep ^processor /proc/cpuinfo | wc -l`
cp liblapack.a libtmglib.a $DEP_PATH/lib
cp librefblas.a $DEP_PATH/lib/libblas.a
cp librefblas.a $DEP_PATH/lib/blas_LINUX.a
$DEP_PATH will be replaced with your local path where you usually keep the dependencies.
Ok, I followed all the steps to get. and this is what I got after compiling octave: [...] checking whether the integer size is correct... no configure: error: BLAS doesn't seem to support 64-bit integers. This is incompatible with --enable-64.
Where did I go wrong? One thing that I did differently was during SuitSparse compilation. There was an error saying that BLAS lib "-lopenblas" could not be found. So I went on suitsparses' config.mk and changed it to BLAS = -lblas And worked fine. So now I really don know what went wrong. Any advice?
This was my configuration for the make.inc
file:
FORTRAN = gfortran
OPTS = -O2 -frecursive -fdefault-integer-8 -fPIC
DRVOPTS = $(OPTS)
NOOPT = -O0 -frecursive -fdefault-integer-8 -fPIC
LOADER = gfortran
LOADOPTS =
This was my configuration for the SuiteSparse SuiteSparse_config/SuiteSparse_config.mk
file:
...
Line 98: INSTALL_LIB = /usr/lib/octave-prefix64/lib
Line 99: INSTALL_INCLUDE = /usr/lib/octave-prefix64/include
...
Line 122: BLAS = -L=/usr/lib/octave-prefix64/lib -lblas -lgfortran
Line 123: LAPACK = -L=/usr/lib/octave-prefix64/lib -llapack
...
Line 133: # OpenBLAS
Line 134: # BLAS = -lopenblas ## **THIS IS IMPORTANT because the original configuration file has this uncommented**
Line 135: # LAPACK =
I noticed that you forgot the -frecursive flag for NOOPTS, so maybe that was the cause of your problem? Try rebuilding/recompiling BLAS/LAPACK and SuiteSparse using this set-up and see if it works for you.
Edit: be sure that lines 98-99 refer to your $prefix64/lib
and $prefix64/include
directories, respectively.
Hi warren, It does not work.
make[2]: Entering directory '/home/terezinha/Downloads/SuiteSparse/KLU/Demo' cc '-DLONGBLAS=long' -O3 -fexceptions -fPIC -I../../SuiteSparse_config -I../../AMD/Include -I../../COLAMD/Include -I../../BTF/Include -I../Include -I../../CHOLMOD/Include kludemo.c -o kludemo ../Lib/libklu.a ../../BTF/Lib/libbtf.a ../../AMD/Lib/libamd.a ../../COLAMD/Lib/libcolamd.a ../../SuiteSparse_config/libsuitesparseconfig.a ../../CHOLMOD/Lib/libcholmod.a -L/usr/lib/octave-prefix64/lib -llapack -L/usr/lib/octave-prefix64/lib -lblas -lgfortran-lblas -lgfortran -lm -lrt /home/terezinha/bin/ld: cannot find -lgfortran-lblas collect2: error: ld returned 1 exit status Makefile:68: recipe for target 'kludemo' failed make[2]: *\ [kludemo] Error 1
Is that the path? This is what I have in my octave lib:
/usr/lib/octave-prefix64/lib$ ls blas_LINUX.a libarpack.la libarpack.so.2 libblas.a libqrupdate.a libqrupdate.so.1 libtmglib.a libarpack.a libarpack.so libarpack.so.2.0.0 liblapack.a libqrupdate.so libqrupdate.so.1.1
I don't have the -lgfotran-lblas flag anywhere in my config file. If you have that in your SuiteSparse_config.mk file, remove it and try again. If you don't have that flag anywhere, can you copy your lines 122 and 123 here so we can see them in detail and help you troubleshoot?
@teresouza I think there was problem in pointing the right path of BLAS in configuring the SuiteSparse. I would suggest recompile the SuiteSparse in following way: make the changes in mk file SuiteSparse_config/SuiteSparse_config.mk
CC = gcc CFLAGS = -fPIC -O -DLP64 -DLONGBLAS='long int' -DLONG='long int' F77 = gfortran F77FLAGS = -fPIC -O -fdefault-integer-8 BLAS = -L$(DEP_PATH)/lib -lblas -lgfortran LAPACK = -L$(DEP_PATH)/lib -llapack
Make sure that you copy the lib files from BLAS SRC folder and placed in $DEP_PATH (according to the local path variable you mentioned)
Ok people, after so many hours I tried to configure octave. I got a lot of warnings (see below). Some things: Most dependencies (cURL, GLPK, etc) are in my $PATH and also isn't that odd that it did not find the COLAMD libraries and UMFPACK?
configure: WARNING: Qhull library not found. This will result in loss of functionality of some geometry functions. configure: WARNING: HDF5 library not found. Octave will not be able to save or load HDF5 data files. configure: WARNING: GLPK library not found. The glpk function for solving linear programs will be disabled. configure: WARNING: cURL library not found. The ftp objects, urlread and urlwrite functions will be disabled. configure: WARNING: OpenGL libs (GL and GLU) not found. Native graphics will be disabled. configure: WARNING: COLAMD library not found. This will result in some lack of functionality for sparse matrices. configure: WARNING: CCOLAMD library not found. This will result in some lack of functionality for sparse matrices. configure: WARNING: CHOLMOD library not found. This will result in some lack of functionality for sparse matrices. configure: WARNING: UMFPACK not found. This will result in some lack of functionality for sparse matrices. configure: WARNING: JAVA_HOME environment variable not initialized. Auto-detection will proceed but is unreliable. configure: WARNING: No javac compiler or jar executable found. Octave will not be able to call Java methods. configure: WARNING: OpenGL libs (GL and GLU) not found -- disabling GUI configure: WARNING: configure: WARNING: You used the EXPERIMENTAL --enable-64 option. configure: WARNING: Are you sure that is what you want to do? configure: WARNING: configure: WARNING: Your Fortran compiler must have an option to generate configure: WARNING: code with 8 byte signed INTEGER values. This option configure: WARNING: should be specified in the F77_INTEGER_8_FLAG variable configure: WARNING: Make. This should work automatically for gfortran. If configure: WARNING: you use another compiler, you will need to set this configure: WARNING: variable on the configure command line. You must also configure: WARNING: compile the ARPACK, BLAS, LAPACK, QRUPDATE, and SuiteSparse configure: WARNING: libraries to use 8 byte signed integers for array indexing. configure: WARNING: configure: WARNING: configure: WARNING: I didn't find the necessary libraries to compile native configure: WARNING: graphics. It isn't necessary to have native graphics, configure: WARNING: but you will need to have gnuplot installed or you won't configure: WARNING: be able to use any of Octave's plotting commands configure: WARNING: configure: configure: NOTE: Libraries or auxiliary programs may be skipped if they are configure: NOTE: not found OR if they are missing required features on your configure: NOTE: system.
EDIT: this is what I used to configure
./configure \ LD_LIBRARY_PATH="/usr/lib/octave-prefix64/lib" \ CPPFLAGS="-I=/usr/lib/octave-prefix64/include" LDFLAGS="-L=/usr/lib/octave-prefix64/lib" \ --enable-64
Contents
terezinha@terezinha-mini-seq:/usr/lib/octave-prefix64/lib$ ls blas_LINUX.a libarpack.so libbtf.a libcholmod.3.0.6.a libcxsparse.a libldl.a librbio.2.2.1.a libsuitesparseconfig.a libamd.2.4.1.a libarpack.so.2 libcamd.2.4.1.a libcholmod.a libklu.1.3.3.a libqrupdate.a librbio.a libtmglib.a libamd.a libarpack.so.2.0.0 libcamd.a libcolamd.2.9.1.a libklu.a libqrupdate.so libspqr.2.0.1.a libumfpack.5.7.1.a libarpack.a libblas.a libccolamd.2.9.1.a libcolamd.a liblapack.a libqrupdate.so.1 libspqr.a libumfpack.a libarpack.la libbtf.1.2.1.a libccolamd.a libcxsparse.3.1.4.a libldl.2.2.1.a libqrupdate.so.1.1 libsuitesparseconfig.4.4.5.a
terezinha@terezinha-mini-seq:/usr/lib/octave-prefix64/include$ ls amd.h cholmod_core.h cholmod_template.h SuiteSparseQR.hpp umfpack.h umfpack_report_status.h umfpack_timer.h btf.h cholmod_function.h colamd.h umfpack_col_to_triplet.h umfpack_load_numeric.h umfpack_report_symbolic.h umfpack_transpose.h camd.h cholmod_gpu.h cs.h umfpack_defaults.h umfpack_load_symbolic.h umfpack_report_triplet.h umfpack_triplet_to_col.h ccolamd.h cholmod_gpu_kernels.h klu.h umfpack_free_numeric.h umfpack_numeric.h umfpack_report_vector.h umfpack_wsolve.h cholmod_blas.h cholmod.h ldl.h umfpack_free_symbolic.h umfpack_qsymbolic.h umfpack_save_numeric.h cholmod_camd.h cholmod_io64.h RBio.h umfpack_get_determinant.h umfpack_report_control.h umfpack_save_symbolic.h cholmod_check.h cholmod_matrixops.h spqr.hpp umfpack_get_lunz.h umfpack_report_info.h umfpack_scale.h cholmod_cholesky.h cholmod_modify.h SuiteSparse_config.h umfpack_get_numeric.h umfpack_report_matrix.h umfpack_solve.h cholmod_complexity.h cholmod_partition.h SuiteSparseQR_C.h umfpack_get_symbolic.h umfpack_report_numeric.h umfpack_symbolic.h cholmod_config.h cholmod_supernodal.h SuiteSparseQR_definitions.h umfpack_global.h umfpack_report_perm.h umfpack_tictoc.h
That is little odd.
Once you check for all dependencies, now it is the time to look for the availability of those module files to the configure script. To get the recent module files which you complied please adjust your LD_LIBRARY_PATH
variable.
export LD_LIBRARY_PATH=$DEP_PATH/lib
Then try to run configure command with following option:
./configure CPPFLAGS="-I$DEP_PATH/include" LIBS="-L$DEP_PATH/lib/ -L/opt/local/lib" LDFLAGS="-L$DEP_PATH/lib/ -L/opt/local/lib" F77=gfortran --prefix=$DEP_PATH/octave --with-glpk-includedir=$DEP_PATH/include --with-glpk-libdir=$DEP_PATH/lib --with-qhull-includedir=$DEP_PATH/include/ --with-qhull-libdir=$DEP_PATH/lib --with-qrupdate-libdir=$DEP_PATH/lib/ --with-qrupdate-includedir=$DEP_PATH/include/ $opts --with-umfpack="-lumfpack -lsuitesparseconfig" --with-umfpack-libdir=$DEP_PATH/lib/ --with-umfpack-includedir=$DEP_PATH/include/
make sure that the above paths are valid and include the necessary dependencies.
Sorry I forgot to mention about the variable $opts
opts="--enable-64"
Hi warren and vipints, just to let you know that octave did not compile. I even reinstalled ubuntu (!) and made everything from scratch, specifying libraries in the last step as vipints suggested, but somehow octave does not seem to find the packages (and I found a lot of threads reporting the same problem as well). Do you have another suggestion of how I can use the tool? I saw a few days ago that it was available at the galaxy server from Ratsch Lab and also Oqtans, but apparently there is a problem with the server (I could not login at the time and now both are down). Do you guys have plans to make it again available anytime soon? (Thanks for the help anyway!)
@teresouza I am sure something went wrong in installing the dependency modules on your local machine that might be the cause for the failure of octave 64bit installation. I have successfully tried this on different flavors of ubuntu and worked well.
I am not seeing any down on our public server, which has been operating normally from past days, didn't see any error in the log message. have you tried on this one https://galaxy.cbio.mskcc.org
?
The lnk works, but all jobs submmted do not run:
An error occurred while getting updates from the server. Please contact a Galaxy administrator if the problem persists.
From: Vipin [mailto:notifications@github.com] Sent: Friday, October 02, 2015 5:29 PM To: ratschlab/rDiff rDiff@noreply.github.com Cc: teresouza terezinhamsouza@gmail.com Subject: Re: [rDiff] rDiff fails on >2K genes during gene expression step; requires 64-bit octave? (my process to build it detailed) (#2)
@teresouza https://github.com/teresouza I am sure something went wrong in installing the dependency modules on your local machine that might be the cause for the failure of octave 64bit installation. I have successfully tried this on different flavors of ubuntu and worked well.
I am not seeing any down on our public server, which has been operating normally from past days, didn't see any error in the log message. have you tried on this one https://galaxy.cbio.mskcc.org?
— Reply to this email directly or view it on GitHub https://github.com/ratschlab/rDiff/issues/2#issuecomment-145062484 . https://github.com/notifications/beacon/AJG7o1a-kDHl90ZnEt0OyjOf16a2vSMYks5o3pojgaJpZM4F0h9M.gif
@teresouza Can you point me from where you are getting this error message.
I checked on our side and able to run jobs on without any issues, data upload also works nicely.
Here a snapshot: http://snag.gy/kMwVW.jpg
From: Vipin [mailto:notifications@github.com] Sent: Friday, October 02, 2015 5:36 PM To: ratschlab/rDiff rDiff@noreply.github.com Cc: teresouza terezinhamsouza@gmail.com Subject: Re: [rDiff] rDiff fails on >2K genes during gene expression step; requires 64-bit octave? (my process to build it detailed) (#2)
@teresouza https://github.com/teresouza Can you point me from where you are getting this error message.
I checked on our side and able to run jobs on without any issues, data upload also works nicely.
— Reply to this email directly or view it on GitHub https://github.com/ratschlab/rDiff/issues/2#issuecomment-145065548 . https://github.com/notifications/beacon/AJG7o2x4nMy7fOrWWuanVUB4FOUjznDhks5o3pvKgaJpZM4F0h9M.gif
Not seeing anything related to this in the Log, I will definitely look in detail.
When did you try to upload the file to the instance? Can you logoff and login then try to upload the file?
Please let me know
The upload was on Tuesday or Wednesday... I've just uploaded a test file now and worked; so I'm uploading a larger file through FTP to see if it works. Thank you!
Once I got rDiff up and running, I attempted to have it analyze my SplAdder results, and it crashed with the following error:
I realized that I needed to re-build Octave as 64-bit, but as this feature is still in an experimental state, it wasn't straightforward. You first need to build the Blas, Lapack, Qrupdate, Arpack, and SuiteSparse libraries as 64-bit aware too. You'll get a reminder warning about this during the build process, whether you do it or not. If you don't do it, though, Octave will likely fail to run when you try to use it (I didn't test that; I'm just making a reasonable assumption).
Here are the webpages detailing instructions on how to install 64-bit octave: initial page: http://timoeckhard.weebly.com/tutorial-install-octave-38-under-ubuntu-1304.html detailed page with step-by-step instructions, tested with gcc 4.3.4 and Debian system: http://www.gnu.org/software/octave/doc/interpreter/Compiling-Octave-with-64_002dbit-Indexing.html
Here are the modifications I made with my set-up, which was tested with Ubuntu 14.04.3, gcc 4.8.4, and Octave 3.8.1 downloaded from ftp://ftp.gnu.org/gnu/octave/octave-3.8.1.tar.gz: Initial Set-up:
/usr/lib/octave-prefix64/
is my location for the$prefix64
directory described in the detailed instructions. All the necessary files go in the$prefix64/lib/
sub-directory, and the Include files from SuiteSparse go into the$prefix64/include/
directory.Blas and Lapack (bundled together): Downloaded lapack-3.5.0; configured the make.inc file (which provides the configuration details to set-up the Makefile) as described; had an error where the Blaslib library was missing (“librefblas.a”), so I first did
make blaslib
beforemake
; then it compiled without errorQrupdate: Downloaded qrupdate-1.1.2; configured the Makeconf file (similarly provides configuration details) as described; you need to add
-L=/usr/lib/octave-prefix64/lib
flag before the-lblas
and-llapack
flags, because these libraries are in a non-standard location now:/usr/lib/octave-prefix64
[substitute this for whatever your $prefix64 directory is]Arpack: Downloaded 3.1.5; added “fdefault-integer-8” to any line where FFLAGS was defined using quotations (the line with just FFLAGS=-g and empty were left alone)
SuiteSparse: Downloaded SuiteSparse 4.4.5 from here, as the URL given in the detailed instructions is dead; changed the SuiteSparse_config/SuiteSparse_config.mk file to reflect the new 64-bit aware library and include directories; could not figure out how to add the suggested code to make shared libraries, so I just did without, and SuiteSparse and Octave appeared to build without issues
Octave: There is no way, as far as I know, to get Octave to build as 64-bit using the Ubuntu package manager
apt-get
. I thus downloaded Octave from the FTP site (link above), and configured it as detailed in the instructions, filling in $prefix64 with my directory path.