Do BP5 files work with ADIOS2.jl?

[lyon-fnal]

Hi, I am using ADIOS2.jl on Cori at NERSC (using adios2 v2.8.2 libraries that I built myself). I can write BP5 files in serial mode (setting the engine type in the config.yaml file), but they seem to be unreadable.

If I try "bpls" on the bp5 file, I get,

[ADIOS2 EXCEPTION] <Core> <Engine> <ThrowUp> : Engine BP5Reader does not support DoAllStepsBlocksInfo

If I try to read the BP5 file with ADIOS2.jl in serial mode, inquire_all_variables returns an empty array.

The tests for adios2 itself pass. I assume that it tries bp5 files (but not really sure).

Have you tried bp5 files using ADIOS2.jl? Thanks! -- Adam

[lyon-fnal]

Ok - I think I figured part of it out. BP5 insists on doing begin_step when reading while BP4 apparently does not. I was able to read a bp5 file with ADIOS2.jl. The bpls problem should go to the adios2 issues list.

By the way, getting ADIOS2.jl to use my build of the adios2 libraries on Cori was a little tricky. I'll open another issue with what I did to make that work so you can comment on it.

[eschnett]

Out of curiosity: Why did you want to use your own build of adios2? The binaries provided by ADIOS2_jll should work there. Did you run into a problem, or was there a feature missing?

[lyon-fnal]

Hi Erik, Thanks for your reply! My understanding is that I need to use the Cray MPI library so that node-to-node communication will use the super-fast network. I configure MPI.jl to use the Cori system MPICH. I think ADIO2_jll will interfere with that.

I would be very happy to be told that I'm wrong - as this configuration is a pain! Please let me know. Thanks, Adam

[lyon-fnal]

I see you are also the author of MPITrampoline. I've never tried that on Cori (I'm using MPI.jl v0.19.2). I suspect that and the newer MPI.jl will make things much easier. I think these things would allow me to use your ADIOS2_jll out of the box. Is this something I can try now?

[eschnett]

Yes, you are right – you currently need to build your own adios2 library for this reason.

This should also work with MPItrampoline, although there are still some rough edges. To do this, you would:

• Build MPIwrapper against Cray MPI
• Use MPI.jl version 0.20.0
• Use MPIPreferences to choose MPItrampoline_jll as binary
• Set a few environment variables at run time that point MPItrampoline to the MPIwrapper library you created

I haven't tried this in the recent past, though.

[lyon-fnal]

OK - thanks - I don't see an MPI v0.20.0 - so probably have to wait for that. I have to look to see if the NERSC folks have set this up already. They now have a common depot, but not sure how MPI is set up.

Thanks again for your help! I'll let you know what I find with ADIOS2.jl on Cori.

[eschnett]

It's the development version of MPI.jl. If you say develop MPI instead of / after add MPI in Julia, you'll get this one.

[lyon-fnal]

Hi Erik, I'm wondering - is there a reason why you didn't implement set_selection? My use case is that I'm going to have several very big separate one dimensional arrays (e.g. a billion rows and one column) in one file. When reading, I may only want to read in part of an array. I think set_selection would let me specify how much of the array I want to read (and where to start). But I don't see it implemented in ADIOS2.jl. If you just haven't gotten around to it, I can try to submit a PR. Or maybe there's something difficult about making that work. Thanks! -- Adam

[eschnett]

This is likely an oversight. It should be straightforward to implement this function. Let me know if you encounter a problem in preparing a PR, I can then assist.

[lyon-fnal]

Ok - thanks! — Adam