trixi2vtk fails for TreeMesh results when running with MPI

[benegee]

I did a Trixi.jl simulation using a TreeMesh and AMR and SaveSolution callbacks. When I try to convert the output files using trixi2vtk, I get:

ERROR: KeyError: key "capacity" not found
Stacktrace:
  [1] getindex(x::HDF5.Attributes, name::String)
    @ HDF5 ~/.julia/packages/HDF5/Ws1wH/src/attributes.jl:374
  [2] #323
    @ ~/.julia/packages/Trixi/16b4d/src/meshes/mesh_io.jl:254 [inlined]
  [3] (::HDF5.var"#17#18"{HDF5.HDF5Context, @Kwargs{}, Trixi.var"#323#340", HDF5.File})()
    @ HDF5 ~/.julia/packages/HDF5/Ws1wH/src/file.jl:101
  [4] task_local_storage(body::HDF5.var"#17#18"{HDF5.HDF5Context, @Kwargs{}, Trixi.var"#323#340", HDF5.File}, key::Symbol, val::HDF5.HDF5Context)
    @ Base ./task.jl:297
  [5] #h5open#16
    @ ~/.julia/packages/HDF5/Ws1wH/src/file.jl:96 [inlined]
  [6] h5open
    @ ~/.julia/packages/HDF5/Ws1wH/src/file.jl:94 [inlined]
  [7] load_mesh_serial(mesh_file::String; n_cells_max::Int64, RealT::Type)
    @ Trixi ~/.julia/packages/Trixi/16b4d/src/meshes/mesh_io.jl:253
  [8] load_mesh_serial
    @ ~/.julia/packages/Trixi/16b4d/src/meshes/mesh_io.jl:246 [inlined]
  [9] macro expansion
    @ ~/.julia/packages/TimerOutputs/RsWnF/src/TimerOutput.jl:237 [inlined]
 [10] trixi2vtk(filename::String; format::Symbol, verbose::Bool, hide_progress::Bool, pvd::Nothing, output_directory::String, nvisnodes::Nothing, save_celldata::Bool, reinterpolate::Bool, data_is_uniform::Bool)
    @ Trixi2Vtk ~/.julia/packages/Trixi2Vtk/rtzWr/src/convert.jl:116
 [11] trixi2vtk(filename::String)
    @ Trixi2Vtk ~/.julia/packages/Trixi2Vtk/rtzWr/src/convert.jl:39
 [12] top-level scope
    @ REPL[2]:1

So, should the h5 files have the "capacity" attribute, or should Trixi not look for it?

It works for non-AMR simulations!

[ranocha]

Can you please post the full stacktrace?

[benegee]

Sure! Just updated it in the original post.

[ranocha]

Did you start with a clean state or may there be some other mesh/solution files?

[benegee]

I started the simulations from scratch and wrote to a fresh output directory each time.

[ranocha]

Please provide the full set of information when describing a bug like this. In this specific case, I guess you ran the simulation with MPI? Does the same problem occur without AMR but with MPI?

[benegee]

Took a while to double check. It is related to MPI, but not to AMR!

I did the following on rocinante (Ubuntu 22.04) using julia 1.10.0, starting from a fresh directory:

JULIA_DEPOT_PATH=./julia-depot julia --project=.
julia> using Pkg
julia> Pkg.add(["Trixi", "Trixi2Vtk", "OrdinaryDiffEq", "MPI"])
julia> using MPI
julia> mpiexec() do cmd
         run(`$cmd -n 2 $(Base.julia_cmd()) --threads=1 --project=@. -e 'using Trixi; trixi_include(default_example())'`)
       end
julia> using Trixi2Vtk
julia> trixi2vtk("out/solution_*.h5")

[benegee]

Maybe this change https://github.com/trixi-framework/Trixi.jl/pull/1748 just needs to be done for

function save_mesh_file(mesh::TreeMesh, output_directory, timestep, mpi_parallel::True)

as well?

[ranocha]

Yes, that looks like it. Would you like to look into it and prepare a PR?