elaird / supy

analyze events stored in TTrees in parallel
8 stars 7 forks source link

fix batch issue seen on screen session #181

Closed gerbaudo closed 10 years ago

gerbaudo commented 11 years ago

These lines make sure that root is started in batch mode, no matter which one is the first module to be pulled in. Without these lines I observe (but not with all analyses) the error below:

X connection to localhost:32.0 broken (explicit kill or server shutdown).

For some reason, which I don't understand, this error seems to show up only when running supy within a screen session (i.e. it doesn't show up with the same analysis running within ssh). However, I can reproduce this error message on lxplus and on other machines (most of them running SLC5).

An alternative solution, maybe less invasive, would be to make sure that one module is always imported before the other ones, and have the SetBatch(True) only there.

Please let me know what you think.

Davide

betchart commented 11 years ago

Hi Davide,

I also use screen frequently (almost exclusively), but haven't had a problem with ROOT trying to start in interactive mode. I usually enter the supy environment via the 'bin/supy' script, which imports ROOT before any other file and has two distinct methods of trying to set batch mode: [0] appending '-b' to the arguments temporarily while initializing ROOT [1] calling gROOT.SetBatch(True) in the default ROOT initialization script

[0] https://github.com/elaird/supy/blob/master/bin/supy#L8 [1] https://github.com/elaird/supy/blob/master/defaults.py#L67

I see that these precautions are probably failing, due to import supy being called in that script on line 6, before setupROOT() is called.

Since I sometimes use __wrappedChain__ and __autoBook__ interactively (they are useful standalone codes), adding SetBatch(True) to those codes would reduce functionality.

I would be in favor of attempting a fix by moving the call to setupROOT(). Please see if the branch #182 'fix-batch-alt' solves your issue.

Burt

gerbaudo commented 11 years ago

Hi Burt,

Thanks a lot for your suggestion. I would prefer a better solution as well, in the direction of what you suggest.

I suspect that I am setting something up differently, either in my code or inherited from the ATLAS setup. I use the bin/supy script as well, and PYTHONPATH with the same commands as you have in susycaf/env.sh.

I cannot reproduce the X connection error with the minimal example, so it's probably due to the order in which I import stuff in my analysis or in my calculables code. I tried to use the branch fix-batch-alt, and I am getting a strange error, and this one I can reproduce both with my analysis and with supy-minimal-example. It's complaining about the missing 'experiment()' function (see below). Does fix-batch-alt work with the CMS setup? I'll keep investigating in the meantime...

Thank you,

Davide

with master

$ cd supy; git checkout master; cd ..
$ supy example_analysis.py --loop 1
site() :  cern
configuration.__file__:  /tmp/gerbaudo/supy-minimal-example/configuration.pyc
configuration.experiment():  atlas
site() :  cern
configuration.__file__:  /tmp/gerbaudo/supy-minimal-example/configuration.pyc
configuration.experiment():  atlas

with fix-batch

$ cd supy; git checkout fix-batch; cd ..
$ supy example_analysis.py --loop 1
site() :  cern
configuration.__file__:  /tmp/gerbaudo/supy-minimal-example/configuration.pyc
configuration.experiment():  atlas
site() :  cern
configuration.__file__:  /tmp/gerbaudo/supy-minimal-example/configuration.pyc
configuration.experiment():  atlas

with fix-batch-alt

$ cd supy/; git checkout fix-batch-alt; cd ..
$ supy example_analysis.py --loop 1
site() :  cern
configuration.__file__:  /tmp/gerbaudo/supy-minimal-example/configuration.pyc
configuration.experiment(): 
Traceback (most recent call last):
  File "/tmp/gerbaudo/supy-minimal-example/supy/bin/supy", line 6, in <module>
    import inspect,configuration
  File "/tmp/gerbaudo/supy-minimal-example/configuration.py", line 1, in <module>
    from supy.defaults import *
  File "/tmp/gerbaudo/supy-minimal-example/supy/__init__.py", line 15, in <module>
    ] : exec("from __%s__ import %s"%(item,item))
  File "<string>", line 1, in <module>
  File "/tmp/gerbaudo/supy-minimal-example/supy/__analysis__.py", line 2, in <module>
    import utils,steps,samples,configuration,calculables,sites
  File "/tmp/gerbaudo/supy-minimal-example/supy/sites/__init__.py", line 76, in <module>
    def info(site = prefix(), key = None) :
  File "/tmp/gerbaudo/supy-minimal-example/supy/sites/__init__.py", line 18, in prefix
    print 'configuration.experiment(): ',configuration.experiment()
AttributeError: 'module' object has no attribute 'experiment'

Although experiment() is defined:

$ grep experiment /tmp/gerbaudo/supy-minimal-example/configuration.py*
/tmp/gerbaudo/supy-minimal-example/configuration.py:def experiment(): return 'atlas'
Binary file /tmp/gerbaudo/supy-minimal-example/configuration.pyc matches
betchart commented 11 years ago

Mmm, yes I see that.

Sorry I didn't test it: it was an github interface hack branch, mostly a suggestion, but I thought it would work.

Playing with it, I narrow the issue to reversing the order of import between supy and configuration. I will investigate a bit further. As a workaround, you may find it easier to set ROOT to batch mode at the top of supy/__init__.py for the moment.

betchart commented 11 years ago

Right. If you want to use the line from supy.defaults import * in your configuration.py, first you have to have imported supy, implicitly even.