Open ashwynsam opened 1 year ago
Thanks a lot for reporting this issue @ashwynsam ! Could you also let us know on which platform or computer cluster this was observed?
@ax3l From the Traceback and the error message
[ADIOS2 EXCEPTION] <Toolkit> <transport::file::FilePOSIX> <CheckFile> : couldn't open file /scratch/users/amsam/soliton_det/potentialknob/diags/diag1/openpmd_000975.bp/data.0, in call to POSIX open: errno = 24: Too many open files
it looks like the error is raised directly by ADIOS2. Have you observed a similar error in the past, on some specific platforms? Also: this seems related to this discussion but I am not sure what the fix is.
@RemiLehe I am running this on Stanford's Sherlock computer (https://www.sherlock.stanford.edu/) .
Thanks for the ping!
This looks like too many open files. Can you print what ulimit -Sn
and ulimit -Hn
output?
You can increase the limit temporarily with ulimit -n ...
to the 2nd value in the command.
But there is a better way. openPMD supports multiple ways to encode in files how to store multiple steps. For convenience, but bad for performance, many codes (incl. WarpX) default to create a new file (with subfiles) per step, which we call file based encoding in openPMD.
An alternative is group based and especially for ADIOS2 variable based encoding, which keeps the numbers of files constant and is also faster. The correspond WarpX option you can try is <diag_name>.openpmd_encoding = v
You will need to rerun your simulation for that.
https://warpx.readthedocs.io/en/latest/usage/parameters.html#full-diagnostics
@ax3l , I get 256 when I run both those commands.
Thanks for the suggestions, I will look into these.
@ax3l , I re ran the sim with <diag_name>.openpmd_encoding = v
and this generated 1 data folder as opposed to 1 for every time step. However, I now don't know how to access the data from this file. Previously, I would do something like :
rho_ions, rho_ions_info = ts.get_field( iteration=10, field='rho_ions', plot=True )
But when I do this, I get this error:
OpenPMDException Traceback (most recent call last)
Cell In [5], line 1
----> 1 rho_ions, rho_ions_info = ts.get_field( iteration=10, field='rho_ions', plot=True )
File ~/miniconda3/envs/ashwyn/lib/python3.10/site-packages/openpmd_viewer/openpmd_timeseries/main.py:488, in OpenPMDTimeSeries.get_field(self, field, coord, t, iteration, m, theta, slice_across, slice_relative_position, plot, plot_range, **kw)
482 raise OpenPMDException(
483 "The requested mode '%s' is not available.\n"
484 "The available modes are: \n - %s" % (m, mode_list))
486 # Find the output that corresponds to the requested time/iteration
487 # (Modifies self._current_i, self.current_iteration and self.current_t)
--> 488 self._find_output(t, iteration)
489 # Get the corresponding iteration
490 iteration = self.iterations[self._current_i]
File ~/miniconda3/envs/ashwyn/lib/python3.10/site-packages/openpmd_viewer/openpmd_timeseries/main.py:629, in OpenPMDTimeSeries._find_output(self, t, iteration)
627 else:
628 iter_list = '\n - '.join([str(it) for it in self.iterations])
--> 629 raise OpenPMDException(
630 "The requested iteration '%s' is not available.\nThe "
631 "available iterations are: \n - %s\n" % (iteration, iter_list))
632 else:
633 pass # self._current_i retains its previous value
OpenPMDException: The requested iteration '10' is not available.
The available iterations are:
- 0
Suggesting that I can only access data from the 0th step when I use ts.get_field()
Can you help me with accessing the data ?
@ax3l More specifically, when using the above workflow (with variable-based encoding), it seems that we are not using the right openPMD-api
interface within openPMD-viewer
:
from openpmd_viewer import OpenPMDTimeSeries
ts = OpenPMDTimeSeries('./diags/openPMD/')
results in:
The opened Series uses variable-based encoding, but is being accessed by
READ_ONLY mode which operates in random-access manner.
Random-access is (currently) unsupported by variable-based encoding
and some iterations may not be found by this access mode.
Consider using Access::READ_LINEAR and Series::readIterations().
@ax3l Do you know how to fix this?
I ran a WarpX simulation with 3000 time steps and I outputted data at every 25 time steps. I want to create a plot of ion density at a particular spatial point vs. time. I first get a 2D slice of the ion density. From this 2D array, I want to collect the data at just the (50,50) cell (this is an arbitrary point). I append this data into a list. Here is the code I used to do this:
I initialized
ts
as :ts = OpenPMDTimeSeries('/scratch/users/amsam/soliton_det/potentialknob/diags/diag1')
However, when I try to run the above for loop, I get the following error:
To debug, I printed out the data at that point in the above loop and I noticed that it runs fine for 39 iterations and then throws the above error. I get this same error when I try to create plots of the 2D data slices at every 25 time steps in my for loop (I am only showing this simpler for loop above to demonstrate this error). Please let me know if you have any insights into how to fix this.
Thank you, Ashwyn