search

elaird / supy

analyze events stored in TTrees in parallel
8 stars 7 forks source link

FNAL batch #160

Closed betchart closed 11 years ago

betchart commented 11 years ago

Hi Ted,

I recently tried the FNAL batch. Submission worked fine, and looping worked fine, but moving the results from the batch nodes was problematic and caused the jobs to fail. I think this might be a new issue due to a recent change of BlueArc filesystems from read/write to read-only. I get an OSError : [Errno 30] Read-only file system.

The exception is raised from 

  File "/uscms/home/bbetchar/cerba/topref/supy/__analysisLooper__.py", line 60, in __call__
    self.moveFiles()
  File "/uscms/home/bbetchar/cerba/topref/supy/__analysisLooper__.py", line 101, in moveFiles
    utils.mkdir(self.globalDir)
  File "/uscms/home/bbetchar/cerba/topref/supy/utils/io.py", line 8, in mkdir
    raise e
OSError: [Errno 30] Read-only file system: '/uscms_data/d2/bbetchar/supy-output//creTopSync/el_sm//ttj_ph'

Could you test the FNAL batch and let me know if you have the same issue, or if it is a problem with my particular setup?

I was looking at the batch documentation and it seems we do a few things they discourage as far as accessing BlueArc (see bad and good examples at page bottom).

elaird commented 11 years ago

Hi Burt,

Yossof (added as a collaborator) has reproduced your problem, and has an idea how to fix it. However, he is held up by https://github.com/elaird/supy/pull/163.

Ted

betchart commented 11 years ago

Addressed by #171