search

pyiron / FAQs

General question board for pyiron users
3 stars 0 forks source link

OSError: Unable to synchronously open file (truncated file: eof = 48, sblock->base_addr = 0, stored_eof = 2048) #49

Closed jan-janssen closed 6 months ago

jan-janssen commented 6 months ago

I presume this might be related to errors with the recent cluster update. But I start to see the following error message: 

2024-04-06 12:50:59,746 - pyiron_log - INFO - run job: job_el_341_8 id: None, status: initialized
Traceback (most recent call last):
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/jobs/job/generic.py", line 763, in run
    self._run_if_new(debug=debug)
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_atomistics/vasp/interactive.py", line 299, in _run_if_new
    super(VaspInteractive, self)._run_if_new(debug=debug)
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/jobs/job/generic.py", line 1281, in _run_if_new
    run_job_with_status_initialized(job=self, debug=debug)
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/jobs/job/runfunction.py", line 91, in run_job_with_status_initialized
    job.save()
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_atomistics/dft/job/generic.py", line 460, in save
    super(GenericDFTJob, self).save()
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/jobs/job/generic.py", line 1138, in save
    self.to_hdf()
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_atomistics/vasp/base.py", line 780, in to_hdf_hdf5/combo529a40377697
    super(VaspBase, self).to_hdf(hdf=hdf, group_name=group_name)
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/jobs/job/interactive.py", line 343, in to_hdf
    super(InteractiveBase, self).to_hdf(hdf=hdf, group_name=group_name)
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/jobs/job/generic.py", line 1094, in to_hdf
    self._hdf5.write_dict(data_dict=self.to_dict())
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/h5io_browser/pointer.py", line 349, in write_dict
    write_dict_to_hdf(
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/h5io_browser/base.py", line 103, in write_dict_to_hdf
    with _open_hdf(file_name, mode="a") as store:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/h5io_browser/base.py", line 158, in _open_hdf
    return h5py.File(name=filename, mode=mode, libver="latest", swmr=swmr)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/h5py/_hl/files.py", line 562, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/h5py/_hl/files.py", line 247, in make_fid
    fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 102, in h5py.h5f.open
OSError: Unable to synchronously open file (truncated file: eof = 48, sblock->base_addr = 0, stored_eof = 2048)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/cli/__main__.py", line 3, in <module>
    main()
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/cli/control.py", line 59, in main
    args.cli(args)
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/cli/wrapper.py", line 37, in main
    job_wrapper_function(
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/jobs/job/wrapper.py", line 186, in job_wrapper_function
    job.run()
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/jobs/job/wrapper.py", line 131, in run
    self.job.run_static()
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/jobs/flex/pythonfunctioncontainer.py", line 96, in run_static
    output = self._function(**self.input.to_builtin())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/ipykernel_14547/1723737464.py", line 11, in combo
  File "/tmp/ipykernel_14547/2935058593.py", line 15, in run_scf
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/utils/deprecate.py", line 171, in decorated
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/jobs/job/generic.py", line 785, in run
    self.drop_status_to_aborted()
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/jobs/job/generic.py", line 1271, in drop_status_to_aborted
    self.project_hdf5["status"] = self.status.string
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/pyiron_base/storage/hdfio.py", line 309, in __setitem__
    _write_hdf5_with_json_support(
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/h5io_browser/base.py", line 257, in _write_hdf5_with_json_support
    _write_hdf(
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/h5io_browser/base.py", line 220, in _write_hdf
    _retry(
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/h5io_browser/base.py", line 438, in _retry
    return func()
           ^^^^^^
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/h5io_browser/base.py", line 221, in <lambda>
    lambda: h5io.write_hdf5(
            ^^^^^^^^^^^^^^^^
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/h5io/_h5io.py", line 172, in write_hdf5
    with h5py.File(fname, mode=mode) as fid:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/h5py/_hl/files.py", line 562, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/cmmc/ptmp/janj/mambaforge/lib/python3.11/site-packages/h5py/_hl/files.py", line 247, in make_fid
    fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 102, in h5py.h5f.open
OSError: Unable to synchronously open file (truncated file: eof = 48, sblock->base_addr = 0, stored_eof = 2048)

Based on https://github.com/h5py/h5py/issues/1101 I tried adding export HDF5_USE_FILE_LOCKING=FALSE to my ~/.bashrc, let's if it helps.

jan-janssen commented 6 months ago

The issue was related to HDF5 files which only had a few bytes in them because my project reached the file system limit.