Closed ChrisRackauckas closed 7 years ago
If you're in the US and want a testing allocation, you can look into getting one from XSEDE:
https://portal.xsede.org/allocations/announcements#trial
The main computers are Stampede and Comet (with the new Bridges being cool too). These would be some good systems to test on (pretty standard setups).
I managed to get Plots.jl and GR working on JURECA (http://www.fz-juelich.de/ias/jsc/EN/Expertise/Supercomputers/JURECA/JURECA_node.html), a supercomputer with 1872 compute nodes based on a CentOS 7 Linux distribution.
These were the necessary steps (I don't have access to the system directories, so i tried an installation in my home directory):
Plots
and GR
packages: julia> Pkg.init()
julia> Pkg.add("Plots)
julia> Pkg.add("GR")
Pkg.checkout("GR")
julia> ENV["GRDIR"]=""
julia> Pkg.build("GR")
INFO: Building GR
INFO: Downloading pre-compiled GR latest Linux binary
julia> using Plots; gr(); histogram(randn(10000))
The above mentioned Linux binary (provided by GR) should work on all RPM based Linux systems. The build script also provides dedicated versions for Ubuntu and Debian. However, problems might occur, if the OS is too old.
@ChrisRackauckas : What is the software archiutecture of the SGE/slurm clusters you are using?
It can depend on the cluster. But I think most of them are SGE/slurm on CentOS. I usually use the XSEDE Bridges, Stampede, Comet, or my local UCI cluster (also CentOS). I'll give this a try and report back.
FWIW unicodeplots has two ascii based canvas
@Evizero How do you use the ASCII canvas with Plots?
For PyPlot, in a headless setup you could use the Agg
backend. (e.g. set ENV["MPLBACKEND"]="Agg"
)
I can't seem to get the headless PyPlot to work @stevengj. This is right after doing the suggested install:
julia> using Plots
julia> ENV["MPLBACKEND"]="Agg"
"Agg"
julia> pyplot()
INFO: Recompiling stale cache file /home/crackauc/.julia/lib/v0.5/PyCall.ji for module PyCall.
INFO: Recompiling stale cache file /home/crackauc/.julia/lib/v0.5/PyPlot.ji for module PyPlot.
WARNING: Couldn't initialize pyplot. (might need to install it?)
INFO: To do a standard install of pyplot, copy and run this:
if !Plots.is_installed("PyPlot")
Pkg.add("PyPlot")
end
withenv("PYTHON" => "") do
Pkg.build("PyPlot")
end
ERROR: InitError: PyError (:PyImport_ImportModule) <type 'exceptions.ImportError'>
ImportError("/lib64/libz.so.1: version `ZLIB_1.2.3.4' not found (required by /home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/../../../libpng16.so.16)",)
File "/home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/pyplot.py", line 29, in <module>
import matplotlib.colorbar
File "/home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/colorbar.py", line 34, in <module>
import matplotlib.collections as collections
File "/home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/collections.py", line 27, in <module>
import matplotlib.backend_bases as backend_bases
File "/home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 62, in <module>
import matplotlib.textpath as textpath
File "/home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/textpath.py", line 18, in <module>
from matplotlib.mathtext import MathTextParser
File "/home/crackauc/.julia/v0.5/Conda/deps/usr/lib/python2.7/site-packages/matplotlib/mathtext.py", line 60, in <module>
import matplotlib._png as _png
in pyerr_check at /home/crackauc/.julia/v0.5/PyCall/src/exception.jl:56 [inlined]
in pyerr_check at /home/crackauc/.julia/v0.5/PyCall/src/exception.jl:61 [inlined]
in macro expansion at /home/crackauc/.julia/v0.5/PyCall/src/exception.jl:81 [inlined]
in pyimport(::String) at /home/crackauc/.julia/v0.5/PyCall/src/PyCall.jl:387
in __init__() at /home/crackauc/.julia/v0.5/PyPlot/src/PyPlot.jl:235
in _include_from_serialized(::String) at ./loading.jl:150
in _require_from_serialized(::Int64, ::Symbol, ::String, ::Bool) at ./loading.jl:187
in require(::Symbol) at ./loading.jl:394
in _initialize_backend(::Plots.PyPlotBackend) at /home/crackauc/.julia/v0.5/Plots/src/backends/pyplot.jl:63
in backend() at /home/crackauc/.julia/v0.5/Plots/src/backends.jl:151
in #pyplot#137(::Array{Any,1}, ::Function) at /home/crackauc/.julia/v0.5/Plots/src/backends.jl:20
in pyplot() at /home/crackauc/.julia/v0.5/Plots/src/backends.jl:20
during initialization of module PyPlot
@jheinen
julia> ENV["GRDIR"]=""
""
julia> Pkg.build("GR")
INFO: Building GR
It looks like that isn't downloading the pre-compiled binary for XSEDE Comet. Is there a way to force it?
@ChrisRackauckas
Hmm, setting GRDIR to an empty strong is the way to force it.
See file GR/deps/build.jl
:
import Compat
if "GRDIR" in keys(ENV)
have_dir = length(ENV["GRDIR"]) > 0
elseif ...
end
if !have_dir
...
tarball = "gr-$version-$os-$arch.tar.gz"
if !isfile("downloads/$tarball")
info("Downloading pre-compiled GR $version $os binary")
mkpath("downloads")
download("http://gr-framework.org/downloads/$tarball", "downloads/$tarball")
if os == :Windows
...
else
run(`tar xzf downloads/$tarball`)
rm("downloads/$tarball")
end
end
end
Is there already an older tar file in the downloads folder? On some clusters, the download may hang due to firewall restrictions. Can you download the file manually?
julia> download("http://gr-framework.org/downloads/gr-latest-Linux-x86_64.tar.gz")
Yeah, that's command causes a download. There was no tar in the GR/deps/download folder though.
@ChrisRackauckas : Could you - if you have time - debug the deps/build.j
script? On our clusters it works and I have no access to your machines. Thanks in advance ...
In case you just want to prove GR, simply unpack the tar file in the GR deps
directory and you can probably test Plots :-)
I unpacked the library but got:
ERROR: error compiling #plot_data#4: error compiling clearws: could not load library "/home/crackauc/.julia/v0.5/GR/deps/gr/lib/libGR.so"
/lib64/libc.so.6: version `GLIBC_2.14' not found (required by /home/crackauc/.julia/v0.5/GR/deps/gr/lib/libGR.so)
in (::GR.jlgr.#kw##plot_data)(::Array{Any,1}, ::GR.jlgr.#plot_data) at ./<missing>:0
in #plot#20(::Array{Any,1}, ::Function, ::Array{Float64,1}, ::Vararg{Array{Float64,1},N}) at /home/crackauc/.julia/v0.5/GR/src/jlgr.jl:848
in plot(::Array{Float64,1}, ::Vararg{Array{Float64,1},N}) at /home/crackauc/.julia/v0.5/GR/src/jlgr.jl:840
in #plot#6(::Array{Any,1}, ::Function, ::Array{Float64,1}, ::Vararg{Array{Float64,1},N}) at /home/crackauc/.julia/v0.5/GR/src/GR.jl:1244
in plot(::Array{Float64,1}, ::Vararg{Array{Float64,1},N}) at /home/crackauc/.julia/v0.5/GR/src/GR.jl:1244
when calling GR directly.
@ChrisRackauckas
What is the output of ldd deps/gr/lib/libGR.so
?
[crackauc@comet-ln3 GR]$ ldd deps/gr/lib/libGR.so
deps/gr/lib/libGR.so: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by deps/gr/lib/libGR.so)
linux-vdso.so.1 => (0x00007ffc031f2000)
libXt.so.6 => /usr/lib64/libXt.so.6 (0x00007f8303711000)
libX11.so.6 => /usr/lib64/libX11.so.6 (0x00007f83033d4000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f83031b7000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f8302fb2000)
libc.so.6 => /lib64/libc.so.6 (0x00007f8302c1e000)
libm.so.6 => /lib64/libm.so.6 (0x00007f830299a000)
libSM.so.6 => /usr/lib64/libSM.so.6 (0x00007f8302791000)
libICE.so.6 => /usr/lib64/libICE.so.6 (0x00007f8302575000)
libxcb.so.1 => /usr/lib64/libxcb.so.1 (0x00007f8302357000)
/lib64/ld-linux-x86-64.so.2 (0x0000003cef600000)
libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f8302152000)
libXau.so.6 => /usr/lib64/libXau.so.6 (0x00007f8301f4f000)
For reference, gcc is available and the available modules are:
[crackauc@comet-ln3 GR]$ module avail
-------------------------------------------------------------------------------- /opt/modulefiles/applications/.intel --------------------------------------------------------------------------------
atlas/3.10.2(default) gsl/1.16 ipm/2.0.3(default) netcdf/4.3.2(default) pdt/3.20(default) sprng/2.0b(default) trilinos/11.12.1(default)
boost/1.55.0(default) gsl/2.1(default) lapack/3.6.0(default) p3dfft/2.7.4(default) petsc/3.6.3(default) sundials/2.6.2(default)
fftw/2.1.5 hdf4/2.11(default) mxml/2.9(default) papi/5.4.1(default) scalapack/2.0.2(default) superlu/4.2(default)
fftw/3.3.4(default) hdf5/1.8.14(default) netcdf/3.6.2 parmetis/4.0.3(default) slepc/3.6.2(default) tau/2.23(default)
------------------------------------------------------------------------------------ /opt/modulefiles/mpi/.intel -------------------------------------------------------------------------------------
intelmpi/2016.3.210(default) mvapich2_gdr/2.1(default) mvapich2_ib/2.1(default) openmpi_ib/1.8.4(default)
------------------------------------------------------------------------------------- /opt/modulefiles/compilers -------------------------------------------------------------------------------------
cilk/5.4.6(default) gnu/4.9.2(default) intel/2013_sp1.2.144(default) intel/2016.3.210 pgi/15.7(default)
cmake/3.3.2(default) guile/2.0.11(default) intel/2015.2.164 mono/3.12.0(default) python/2.7.10(default)
----------------------------------------------------------------------------------- /opt/modulefiles/applications ------------------------------------------------------------------------------------
abaqus/6.11-2 bowtie2/2.2.7(default) gamess/2014.12(default) lammps/20160216(default) node/0.12.6(default) singularity/2.1.2(default)
abaqus/6.14-1(default) bwa/0.7.13(default) garli/2.01(default) llvm/3.6.2(default) nwchem/6.6(default) soapdenovo/240(default)
abinit/7.10.5(default) bx-python/0.7.3(default) gateway-usage-reporting/1.0 mafft/7.187(default) octave/4.0.0(default) SOAPsnp/1.03(default)
abyss/1.5.2(default) celera/8.3rc2(default) gaussian/09.D.01(default) matlab/2016a(default) openbabel/2.3.2(default) spades/3.9.0(default)
amber/15(default) cp2k/3.0(default) gaussian/09.E.01 matt/1.00(default) picard/2.3.0(default) spark/1.2.0
apbs/1.4.2(default) cpmd/3.17.1(default) gdal/1.11.2(default) migrate/3.6.11(default) plink/1.9(default) spark/1.5.2(default)
bamtools/2.4.0(default) cuda/6.5 GenomeAnalysisTK/3.5 migrate/3.6.8 polymake/2.14(default) squid/1.9g(default)
bbcp/14.09.02.00.0(default) cuda/7.0(default) GenomeAnalysisTK/3.6(default) miRDeep2/0.0.7(default) proj/4.9.1(default) stacks/1.37(default)
bbftp/3.2.1(default) cufflinks/2.2.1(default) geos/3.4.2(default) miso/0.5.3(default) pysam/0.8.4(default) tophat/2.1.1(default)
bcftools/1.3(default) ddt/6.0.6(default) globus/5.2.5 mkl/11.1.2.144(default) qchem/4.3.2(default) trimmomatic/0.35(default)
beagle/2.1(default) dendropy/4.0.3(default) gmap_gsnap/20151231(default) mkl/11.2.2.164 qe/5.3.0(default) trinity/2.1.1(default)
beast/1.8.0 diamond/0.7.12(default) gmp/6.0.0a(default) mkl/11.3.3 qiime/1.9.1(default) vasp/4.6
beast/1.8.1 drive-data/6.2(default) gnutools/2.69(default) molden/5.0.7(default) R/3.2.3(default) vasp/5.4.1.20160205(default)
beast/1.8.2(default) edena/3.131028(default) grace/5.1.25(default) mopac/2016(default) randfold/2.0(default) vcftools/0.1.14(default)
beast2/2.1.3(default) eigen/3.2.7(default) gromacs/5.1.2(default) mpc/1.0.3(default) rapidminer/7.1.0(default) velvet/1.2.10(default)
bedtools/2.25.0(default) eman2/2.1(default) hadoop/1.2.1 mpfr/3.1.2(default) raxml/8.2.3(default) ViennaRNA/2.2.4(default)
biopython/1.66(default) emboss/6.5.7(default) hadoop/2.6.0(default) mpi4py/1.3.1(default) relion/1.4(default) visit/2.9.1(default)
biotools/4(default) fastqc/0.11.4(default) hmmer/3.1b2(default) mpiblast/1.6.0(default) rseqc/2.6.3(default) vmd/1.9.2(default)
bismark/0.15.0(default) fasttree/2.1.8(default) htseq/0.6.1p1(default) namd/2.10(default) samtools/1.2 weka/3.7.12(default)
blast/2.3.0(default) fastx/0.0.14(default) idba-ud/1.1.1(default) namd/2.9 samtools/1.3(default)
blat/35(default) frealign/9.11(default) idl/8.4(default) ncl_ncarg/6.2.1(default) scipy/2.7(default)
bowtie/1.1.2(default) fsa/1.15.9(default) jasper/1.900.1(default) ncview/2.1.7(default) siesta/3.2.6(default)
----------------------------------------------------------------------------------- /usr/share/Modules/modulefiles -----------------------------------------------------------------------------------
dot module-git module-info modules null use.own
@ChrisRackauckas : Hmm, so glibc
is too old? What is the output of rpm -q -f /lib64/libc.so.6
and cat /etc/redhat-release
?
[crackauc@comet-ln3 GR]$ rpm -q -f /lib64/libc.so.6
glibc-2.12-1.166.el6_7.7.x86_64
[crackauc@comet-ln3 GR]$ cat /etc/redhat-release
CentOS release 6.7 (Final)
@ChrisRackauckas - Ok. I think we should better deploy the CentOS 6 build for the GR run-time. Right now, we use CentOS 7 for building the "pre-compiled GR latest Linux binary".
I will pre-check V6 compatibility with V7 (and other RPM-based Linux distributions) and tell you about the results when I'm back in office. The julia
binary itself has similar dependency, so I expect no problems.
Thanks for testing!
The GR changes plus https://github.com/tbreloff/Plots.jl/issues/578 works really well. I am able to see the plots with UnicodePlots, re-make and save them using GR, and then send them back to my computer even without X11 forwarding. With X11 forwarding GR works fine. This makes a good workflow for plotting on common clusters. I've tried this now on 3 clusters with the same results: GR/Unicode works fine, Plotly(JS) of course doesn't because of the lack of web browsers, and PyPlot still has issues.
I think this should be added to the documentation on backends. What exactly do you think it should say. I want to say something like "Compatible with clusters based on CentOS 6/7 based SGE and Slurm clusters".
The PyPlot problem is due to multiple incompatible libz libraries: https://github.com/JuliaPy/PyPlot.jl/issues/151
Does anyone have a way to get Plots.jl backends working on standard clusters? Any of them? I am talking about SGE/slurm clusters (likely accessed via SSH, X11 forwarding sometimes enabled, like XSEDE Comet, Bridges, etc.) where some standard components like Python, matplotlib, GTK, etc. are contained in (non-Julia) modules which you have to import. I've done the "native" thing of
module avail
for any module which makes sense, and running the Julia code, but Plots.jl can't find a backend. As of right now, I don't know of a way to even get plots to save. Here's the lowdown:Anyone have a working method for Plots.jl on a cluster like this?