no output plots

[tiborsimko]

As mentioned IRL, the example stopped producing plots if one rebuilds images and reruns the example from scratch using vanilla yadage. (I.e. also outside of REANA.)

How to reproduce:

$ docker rmi atlas/analysisbase
$ cd eventselection
$ docker build -t reanahub/reana-demo-atlas-recast-eventselection . --no-cache
$ cd ../statanalysis
$ docker build -t reanahub/reana-demo-atlas-recast-statanalysis .
$ cd ..
$ mkvirtualenv yadage27 -p /usr/bin/python2.7
$ yadage-run _run workflow/workflow.yml ./workflow/test_inputs/inp1.yml

The ROOT files were produced:

$ find _run | grep root
_run/eventselection/submitDir/input/sample.root
_run/eventselection/submitDir/hist-sample.root
_run/eventselection/submitDir/hist/sample.root
_run/eventselection/submitDir/driver.root

as well as the JSON fit results:

$ find _run | grep limit
_run/statanalysis/fitresults/limit_data.json
_run/statanalysis/fitresults/limit_data_nomsignal.json

but no plots:

$ find _run | grep png

The statanalysis logs show that:

$ tail -10 _run/statanalysis/_packtivity/statanalysis.run.log
2019-11-28 11:38:51,883 | pack.statanalysis.ru |   INFO | NO.   NAME      VALUE            ERROR          SIZE      DERIVATIVE
2019-11-28 11:38:51,883 | pack.statanalysis.ru |   INFO | 1  alpha_syst1  -7.70828e-01   9.50708e-01   4.66492e-03  -4.71670e-03
2019-11-28 11:38:51,883 | pack.statanalysis.ru |   INFO | ERR DEF= 0.5
2019-11-28 11:38:51,884 | pack.statanalysis.ru |   INFO | AsymptoticCalculator::EvaluateNLL -  value = -1210.79 for poi fixed at = 2    fit time : Real time 0:00:00, CP time 0.000
2019-11-28 11:38:51,884 | pack.statanalysis.ru |   INFO | [#0] PROGRESS:Eval --          ASIMOV data qmu_A = 16.9355 condNLL = -1210.79 uncond -1219.26
2019-11-28 11:38:51,884 | pack.statanalysis.ru |   INFO | [#0] PROGRESS:Eval -- poi = 2 qmu = 9.06996 qmu_A = 16.9355 sigma = 0.485994  CLsplusb = 0.00129921 CLb = 0.865125 CLs = 665.885
2019-11-28 11:38:51,918 | pack.statanalysis.ru |   INFO | cling::DynamicLibraryManager::loadLibrary(): libtiff.so.5: cannot open shared object file: No such file or directory
2019-11-28 11:38:51,918 | pack.statanalysis.ru |   INFO | Error in <TInterpreter::TCling::AutoLoad>: failure loading library libASImage.so for TASImage
2019-11-28 11:38:51,919 | pack.statanalysis.ru |   INFO | cling::DynamicLibraryManager::loadLibrary(): libtiff.so.5: cannot open shared object file: No such file or directory
2019-11-28 11:38:52,092 | pack.statanalysis.ru |   INFO | 0.5 0.574709475331

Note the libtiff troubles.

@lukasheinrich :

(1) Is the code of the example recent and how you would like it to look? Perhaps there were some changes on the ATLAS base image side that were not propagated to the REANA repo?

(2) Can we pin the base image and all free dependencies to some particular versions? The example uses currently the latest base image:

eventselection/Dockerfile:1:FROM atlas/analysisbase:latest
statanalysis/Dockerfile:1:FROM atlas/analysisbase

as well as ian unpinned htfools library:

statanalysis/Dockerfile:3:RUN sudo sh -c "source /home/atlas/release_setup.sh && pip install hftools"

This is prone to failing sooner or later... For a nicely complete analysis preservation example, it would be good to use pinned versions of all dependencies.

Can you please have a look and advise which versions to use?

[lukasheinrich]

this should be fixed by tagging the image of the stat. analysis to be atlas/analysisbase:21.2.51

[tiborsimko]

Thanks, it works, I've pushed new images to Docker Hub and I'll make a PR.

P.S. Since we use an imposed atlas user, one has to test locally as follows:

$ rm -rf _run && mkdir -p _run/eventselection _run/statanalysis/resultdir && chmod -R a+rwx _run
$ yadage-run _run workflow/workflow.yml ./workflow/test_inputs/inp1.yml

in between runs.