Demonstrate ADIOS output on Frontier

[sethrj]

@frs69wq Could you input (as a comment) the plan of execution you told me two weeks ago?

[frs69wq]

• Port current version of ADIOS writer to the develop branch [DONE]
• Determine how to the add ADIOS to Celeritas' build chain on Frontier
  • Check with Norbert what is currently available (Spack, E4S, other)
  • Check what is available for the standard package and for the GPU-aware version.
• Deployment and tests on Frontier
• Implement/test a version that writes data directly from the GPU
• Reduce size of ADIOS metadata by writing every X iterations of the Step writer
  • Make X a configuration parameter
  • Assess what are the implications in terms of additional temporary memory footprint and memory copies
  • Assess what are the implications when data is directly fetched from the GPU
• Implement a staging/streaming version with an external ADIOS process
  • stream data from Celeritas to this ADIOS process
  • Aggregate data across multiple (Celeritas) steps
  • Use Adios to write to file system
• Combine one Celeritas instance and external ADIOS writer in the same MPI program
• Combine multiple Celeritas instances and an ADIOS writer in a single MPI (fan-out style) program
  • Can we do better in terms of data aggregation across instances?
• Perform some scalability tests on Frontier
  • using gnu-parallel (w/ Ketan) to launch multiple MPI(Celeritas+ADIOS) instances.