How to use python binding to modify an existing bp file?

[liangwang0734]

I tried fh = adios2.open(filename, "a") for "append" mode on a bp v3 file, but the returned file handler indicates an empty file.

I would like to modify values in a variable, say fh['density'] *= 2 or fh['densit'][...] = np.array(density_array)

PS: I also tried adios-python from adios 1.13, but couldn't figure out a working example, either.

[williamfgc]

@liangwang0734 append mode is not supported in BP3. I might probably need to add an exception. Try BP4. In general, adios doesn't modify existing data, but appends to it.

[liangwang0734]

Thanks, @williamfgc

I'm trying to create a restart file for a code that currently uses adios 1.13. After your clarification that bp3 would not support "append" mode, I ended up creating a bp3 new file (with engine_type='bp3' in open).

However, through the python bindings, I was only to create the bp file with associated folder, while in the code I use, the bp3 files are single files.

In small tests, the code was able to read the bp3 file and folder. But in big runs, reading the generated bp file and folder dies from out-of-memory.

Question: Is there a way to generate the single bp3 file without the folder through the python binding (or cpp binding)?

[williamfgc]

Question: Is there a way to generate the single bp3 file without the folder through the python binding (or cpp binding)?

@liangwang0734 single file = metadata + data was never supported in adios2. It was decided that generating a directory is the way forward (for BP4). Also, looking into the code (thanks for sharing), it might be worth to migrate to adios2 as it's actively supported. The latter will eliminate a lot of boilerplate in your code. Hope it helps.