Loading files over XRootD protocol

[alexpearce]

I'm unable to load files with ROOT over the [XRootD]() protocol. Below is what I do, and then what I think the problem might be.

Reproducing

I create a conda environment with:

$ curl -O https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
$ bash Miniconda3-latest-Linux-x86_64.sh
$ conda config --add channels https://conda.anaconda.org/NLeSC
$ conda create --name=exampleenv root=6 python=3

And then I try to load a file over XRootD:

$ root -l -b root://eoslhcb.cern.ch//eos/lhcb/grid/user//lhcb/user/a/apearce/2016_09/139512/139512225/DVntuple.root

I first get an error about loading libpcre:

/afs/cern.ch/user/a/apearce/miniconda3/pkgs/root-6.04-py3.4_gcc4.8.2/bin/root.exe: error while loading shared libraries: libpcre.so.1: cannot open shared object file: No such file or directory

I modify my LD_LIBRARY_PATH and try again:

$ LD_LIBRARY_PATH=/afs/cern.ch/user/a/apearce/miniconda3/envs/exampleenv/lib root -l -b root://eoslhcb.cern.ch//eos/lhcb/grid/user//lhcb/user/a/apearce/2016_09/139512/139512225/DVntuple.root
Error in <TUnixSystem::FindDynamicLibrary>: Netx[.so | .dll | .dylib | .sl | .dl | .a] does not exist in /afs/cern.ch/user/a/apearce/miniconda3/pkgs/root-6.04-py3.4_gcc4.8.2/lib:/afs/cern.ch/user/a/apearce/miniconda3/envs/charmpol/lib:.:/afs/cern.ch/user/a/apearce/miniconda3/pkgs/root-6.04-py3.4_gcc4.8.2/lib:/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64
root [0]
Attaching file root://eoslhcb.cern.ch//eos/lhcb/grid/user//lhcb/user/a/apearce/2016_09/139512/139512225/DVntuple.root as _file0...
Error in <TUnixSystem::FindDynamicLibrary>: Netx[.so | .dll | .dylib | .sl | .dl | .a] does not exist in /afs/cern.ch/user/a/apearce/miniconda3/pkgs/root-6.04-py3.4_gcc4.8.2/lib:/afs/cern.ch/user/a/apearce/miniconda3/envs/charmpol/lib:.:/afs/cern.ch/user/a/apearce/miniconda3/pkgs/root-6.04-py3.4_gcc4.8.2/lib:/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64
Error in <TUnixSystem::FindDynamicLibrary>: Netx[.so | .dll | .dylib | .sl | .dl | .a] does not exist in /afs/cern.ch/user/a/apearce/miniconda3/pkgs/root-6.04-py3.4_gcc4.8.2/lib:/afs/cern.ch/user/a/apearce/miniconda3/envs/charmpol/lib:.:/afs/cern.ch/user/a/apearce/miniconda3/pkgs/root-6.04-py3.4_gcc4.8.2/lib:/lib64/tls/x86_64:/lib64/tls:/lib64/x86_64:/lib64:/usr/lib64/tls/x86_64:/usr/lib64/tls:/usr/lib64/x86_64:/usr/lib64
(class TFile *) 0x0
root [1]

I see the same messages if I also conda install xrootd.

Maybe reasons

There is a guide for installing XRootD with ROOT, and it says XRootD can either be installed by ROOT during the make step, or ROOT can be told about an external build. I don't think the NLeSC builds enable either of these options. Would it be possible to do so?

[remenska]

Hi Alex,

There is a ROOT binary with xrootd compiled some time ago. It's not in the main channel, however; it's in a channel/label called xrootd (needs some testing). We should at some point switch to conda features rather than labeling different flavours of ROOT.

https://anaconda.org/nlesc/repo/files?type=all&label=xrootd

[alexpearce]

Woah, cool! That does work. Now I do:

$ conda create --name=exampleenv python=3.4
$ conda install --channel nlesc/label/xrootd root=6

and I can open files over XRootD. I do still have to specify the LD_LIBRARY_PATH, though:

$ LD_LIBRARY_PATH=/afs/cern.ch/user/a/apearce/miniconda3/envs/exampleenv/lib root -l -b

Is that expected?

[remenska]

Cool, thanks for reporting! No, in principle the point of anaconda environments is that you shouldn't specify any additional environmental variables, for things to work...unless of course the conda binary you install requires them :) Like ROOT needs ROOTSYS/PYTHONPATH after sourcing thisroot.sh.

When you activate the environment, do you see the correct:

Activate: ROOT has been sourced. Environment settings are ready. 
ROOTSYS=~/anaconda/envs/testenv

? Or is it this:

Activate: ROOT has been sourced. Environment settings are ready. 
ROOTSYS=

I think I compiled this binary before conda changed its mind as to what env variables are available during install/environment-activation time, so that could be the issue. Another possibility is that XRootD just depends on LD_LIBRARY_PATH (less likely). To be honest, I haven't checked that.

[alexpearce]

Ah, I do get the second one, just ROOTSYS=.

[remenska]

Oke, then we know, thanks. Need to find time to recompile with the recipe changed a bit :) I will keep this issue open as a reminder, but feel free to give it a try too, if you want. I can always upload your binary to the cloud.

Meanwhile sourcing thisroot.sh is a "hack".

[egpbos]

The NLeSC ROOT Conda package is no longer maintained and does not work properly with current Conda, modern Python, or modern ROOT. Use conda-forge ROOT instead:

Installing in a new environment:

conda create -n myrootenv python=3.7 root -c conda-forge
conda activate myrootenv
conda config --env --add channels conda-forge

From then on, use conda activate myrootenv to get in and conda deactivate to get out of the env. You should have a recent version of conda.

or

Installing in the current environment:

conda config --env --add channels conda-forge 
conda install root

Unlike the NLeSC package, this can even be the base environment if you really want to do that (it is not ideal, since it will be rather invasive, adding things like conda's compiler packages that are needed to compile with ROOT and Conda).

Conda-forge ROOT provides ROOT 6.16.00+, and supports Python 3.7, 3.6, and 2.7 on macOS and Linux. It is being tested by the ROOT team, and is automatically build by the conda-forge infrastructure. All default features and a huge number of optional features are turned on, and you can use it anywhere you would use ROOT, JupyROOT, or PyROOT.