multi-node installation seems broken if not shared dir

daniviga commented 5 years ago

Running a virtual cluster using Docker (without shared dir set) I get:

Traceback (most recent call last):
  File "/opt/openquake/lib/python3.6/site-packages/openquake/engine/engine.py", line 347, in run_calc
    close=False, **kw)
  File "/opt/openquake/lib/python3.6/site-packages/openquake/calculators/base.py", line 203, in run
    self.post_execute(self.result)
  File "/opt/openquake/lib/python3.6/site-packages/openquake/calculators/classical.py", line 264, in post_execute
    self.calc_stats(self.hdf5cache)
  File "/opt/openquake/lib/python3.6/site-packages/openquake/calculators/classical.py", line 284, in calc_stats
    ).reduce(self.save_hazard_stats)
  File "/opt/openquake/lib/python3.6/site-packages/openquake/baselib/parallel.py", line 685, in reduce
    return self.submit_all().reduce(agg, acc)
  File "/opt/openquake/lib/python3.6/site-packages/openquake/baselib/parallel.py", line 487, in reduce
    for result in self:
  File "/opt/openquake/lib/python3.6/site-packages/openquake/baselib/parallel.py", line 453, in __iter__
    val = result.get()
  File "/opt/openquake/lib/python3.6/site-packages/openquake/baselib/parallel.py", line 307, in get
    raise etype(msg)

OSError: 
  File "/opt/openquake/lib/python3.6/site-packages/openquake/baselib/parallel.py", line 317, in new
    val = func(*args)
  File "/opt/openquake/lib/python3.6/site-packages/openquake/baselib/parallel.py", line 384, in gfunc
    yield func(*args)
  File "/opt/openquake/lib/python3.6/site-packages/openquake/calculators/classical.py", line 312, in build_hazard_stats
    pgetter.init()  # if not already initialized
  File "/opt/openquake/lib/python3.6/site-packages/openquake/calculators/getters.py", line 69, in init
    self.dstore = hdf5.File(self.dstore, 'r')
  File "/opt/openquake/lib64/python3.6/site-packages/h5py/_hl/files.py", line 312, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
  File "/opt/openquake/lib64/python3.6/site-packages/h5py/_hl/files.py", line 142, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 78, in h5py.h5f.open
OSError: Unable to open file (unable to open file: name = '/home/openquake/oqdata/cache_1.hdf5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

demo used was the AreaSourceClassicalPSHA.zip other demos work as expected (like Event Based PSHA using Area Source)

daniviga commented 5 years ago

See https://travis-ci.org/gem/oq-engine-celery/builds/442659601

daniviga commented 5 years ago

Superseded by #4238

gem / oq-engine

multi-node installation seems broken if not shared dir #4084