Footer is header in vlasiator output

iljah commented 8 years ago

I ran a small test with vlasiator from this week's git and output files seem to start with xml data contrary to documentation. hexdump shows that binary data seems to start only after xml data, although maybe xml was written in the wrong place? I get:

head -n 55 initial-grid.0000000.vlsv 
<VLSV>
 IDS arraysize="160" datasize="4" datatype="uint" mesh="SpatialGrid" name="avgs" vectorsize="1">1528</BLOCKIDS>
   <BLOCKSPERCELL arraysize="1" datasize="4" datatype="uint" mesh="SpatialGrid" name="avgs" vectorsize="1">300</BLOCKSPERCELL>
   <BLOCKVARIABLE arraysize="160" datasize="4" datatype="float" mesh="SpatialGrid" name="avgs" vectorsize="64">2168</BLOCKVARIABLE>
   <CELLSWITHBLOCKS arraysize="1" datasize="8" datatype="uint" mesh="SpatialGrid" name="avgs" vectorsize="1">292</CELLSWITHBLOCKS>
   <MESH arraysize="1" datasize="8" datatype="uint" name="SpatialGrid" type="multi_ucd" vectorsize="1" xperiodic="yes" yperiodic="yes" zperiodic="yes">276</MESH>
   <MESH_BBOX arraysize="6" datasize="8" datatype="uint" mesh="SpatialGrid" vectorsize="1">16</MESH_BBOX>
   <MESH_BBOX arraysize="6" datasize="8" datatype="uint" mesh="avgs" type="amr_ucd" vectorsize="1">304</MESH_BBOX>
   <MESH_DOMAIN_SIZES arraysize="1" datasize="4" datatype="uint" mesh="SpatialGrid" vectorsize="2">284</MESH_DOMAIN_SIZES>
   <MESH_GHOST_DOMAINS arraysize="0" datasize="8" datatype="uint" mesh="SpatialGrid" vectorsize="1">292</MESH_GHOST_DOMAINS>
   <MESH_GHOST_LOCALIDS arraysize="0" datasize="8" datatype="uint" mesh="SpatialGrid" vectorsize="1">292</MESH_GHOST_LOCALIDS>
   <MESH_NODE_CRDS_X arraysize="2" datasize="8" datatype="float" mesh="SpatialGrid" vectorsize="1">64</MESH_NODE_CRDS_X>
   <MESH_NODE_CRDS_X arraysize="49" datasize="8" datatype="float" mesh="avgs" type="amr_ucd" vectorsize="1">352</MESH_NODE_CRDS_X>
   <MESH_NODE_CRDS_Y arraysize="2" datasize="8" datatype="float" mesh="SpatialGrid" vectorsize="1">80</MESH_NODE_CRDS_Y>
   <MESH_NODE_CRDS_Y arraysize="49" datasize="8" datatype="float" mesh="avgs" type="amr_ucd" vectorsize="1">744</MESH_NODE_CRDS_Y>
   <MESH_NODE_CRDS_Z arraysize="2" datasize="8" datatype="float" mesh="SpatialGrid" vectorsize="1">96</MESH_NODE_CRDS_Z>
   <MESH_NODE_CRDS_Z arraysize="49" datasize="8" datatype="float" mesh="avgs" type="amr_ucd" vectorsize="1">1136</MESH_NODE_CRDS_Z>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="time" vectorsize="1">120</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="dt" vectorsize="1">128</PARAMETER>
   <PARAMETER arraysize="1" datasize="4" datatype="uint" name="timestep" vectorsize="1">136</PARAMETER>
   <PARAMETER arraysize="1" datasize="4" datatype="int" name="fieldSolverSubcycles" vectorsize="1">140</PARAMETER>
   <PARAMETER arraysize="1" datasize="4" datatype="uint" name="fileIndex" vectorsize="1">144</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="xmin" vectorsize="1">148</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="xmax" vectorsize="1">156</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="ymin" vectorsize="1">164</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="ymax" vectorsize="1">172</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="zmin" vectorsize="1">180</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="zmax" vectorsize="1">188</PARAMETER>
   <PARAMETER arraysize="1" datasize="4" datatype="uint" name="xcells_ini" vectorsize="1">196</PARAMETER>
   <PARAMETER arraysize="1" datasize="4" datatype="uint" name="ycells_ini" vectorsize="1">200</PARAMETER>
   <PARAMETER arraysize="1" datasize="4" datatype="uint" name="zcells_ini" vectorsize="1">204</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="vxmin" vectorsize="1">208</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="vxmax" vectorsize="1">216</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="vymin" vectorsize="1">224</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="vymax" vectorsize="1">232</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="vzmin" vectorsize="1">240</PARAMETER>
   <PARAMETER arraysize="1" datasize="8" datatype="float" name="vzmax" vectorsize="1">248</PARAMETER>
   <PARAMETER arraysize="1" datasize="4" datatype="uint" name="vxblocks_ini" vectorsize="1">256</PARAMETER>
   <PARAMETER arraysize="1" datasize="4" datatype="uint" name="vyblocks_ini" vectorsize="1">260</PARAMETER>
   <PARAMETER arraysize="1" datasize="4" datatype="uint" name="vzblocks_ini" vectorsize="1">264</PARAMETER>
   <PARAMETER arraysize="1" datasize="4" datatype="uint" name="max_velocity_ref_level" vectorsize="1">268</PARAMETER>
   <PARAMETER arraysize="1" datasize="4" datatype="float" name="version" vectorsize="1">272</PARAMETER>
   <VARIABLE arraysize="1" datasize="8" datatype="uint" mesh="SpatialGrid" name="CellID" vectorsize="1">112</VARIABLE>
   <VARIABLE arraysize="1" datasize="8" datatype="float" mesh="SpatialGrid" name="B" vectorsize="3">43128</VARIABLE>
   <VARIABLE arraysize="1" datasize="8" datatype="float" mesh="SpatialGrid" name="background_B" vectorsize="3">43152</VARIABLE>
   <VARIABLE arraysize="1" datasize="4" datatype="uint" mesh="SpatialGrid" name="Blocks" vectorsize="1">43176</VARIABLE>
   <VARIABLE arraysize="1" datasize="8" datatype="float" mesh="SpatialGrid" name="E" vectorsize="3">43180</VARIABLE>
   <VARIABLE arraysize="1" datasize="4" datatype="int" mesh="SpatialGrid" name="MPI_rank" vectorsize="1">43204</VARIABLE>
   <VARIABLE arraysize="1" datasize="8" datatype="float" mesh="SpatialGrid" name="PTensorDiagonal" vectorsize="3">43208</VARIABLE>
   <VARIABLE arraysize="1" datasize="8" datatype="float" mesh="SpatialGrid" name="PTensorOffDiagonal" vectorsize="3">43232</VARIABLE>
   <VARIABLE arraysize="1" datasize="8" datatype="float" mesh="SpatialGrid" name="perturbed_B" vectorsize="3">43256</VARIABLE>
   <VARIABLE arraysize="1" datasize="8" datatype="float" mesh="SpatialGrid" name="Pressure" vectorsize="1">43280</VARIABLE>
   <VARIABLE arraysize="1" datasize="8" datatype="float" mesh="SpatialGrid" name="rho" vectorsize="1">43288</VARIABLE>
   <VARIABLE arraysize="1" datasize="8" datatype="float" mesh="SpatialGrid" name="rho_v" vectorsize="3">43296</VARIABLE>
</VLSV>

with this config file:

propagate_field = 1
propagate_vlasov_acceleration = 1
propagate_vlasov_translation = 1
dynamic_timestep = 1
project = Testiih

[io]
diagnostic_write_interval = 1
write_initial_state = 1
restart_walltime_interval = 100000
number_of_restarts = 1

system_write_t_interval = 1
system_write_file_name = fullf
system_write_distribution_stride = 1
system_write_distribution_xline_stride = 0
system_write_distribution_yline_stride = 0
system_write_distribution_zline_stride = 0

[gridbuilder]
x_length = 1
y_length = 1
z_length = 1
x_min = 0.0
x_max = 1.0e3
y_min = 0.0
y_max = 1.0e3
z_min = 0.0
z_max = 1.0e3
vx_min = -2.0e6
vx_max = +2.0e6
vy_min = -2.0e6
vy_max = +2.0e6
vz_min = -2.0e6
vz_max = +2.0e6
vx_length = 12
vy_length = 12
vz_length = 12
t_max = 10.0
dt = 5.0

[vlasovsolver]
#minCFL = 0.4
#maxCFL = 0.6
#vlasovSemiLagAcceleration = 0

[boundaries]
periodic_x = yes
periodic_y = yes
periodic_z = yes

[variables]
output = Rho
output = B
output = BackgroundB
output = PerturbedB
output = Pressure
output = RhoV
output = E
output = PTensor
output = MPIrank
output = Blocks
diagnostic = Blocks
diagnostic = RhoLossAdjust
diagnostic = RhoLossVelBoundary
diagnostic = MaxDistributionFunction
diagnostic = MinDistributionFunction

[sparse]
minValue = 1.0e-20

[Testiih]
n = 2

Vx = -5.0e5
Vy = 0.0
Vz = 0.0
Tx = 1.0e5
Ty = 1.0e5
Tz = 1.0e5
rho  = 1.0e6
rhoPertAbsAmp = 0.0

Vx = 5.0e5
Vy = 0.0
Vz = 0.0
Tx = 1.0e5
Ty = 5.0e5
Tz = 5.0e5
rho = 1.0e4
rhoPertAbsAmp = 0.0

Bx = 1e-12
By = 1e-11
Bz = 1e-10

dBx = 0.0
dBy = 0.0
dBz = 0.0

magXPertAbsAmp = 0.0
magYPertAbsAmp = 0.0
magZPertAbsAmp = 0.0

lambda = 1

nVelocitySamples = 2

I started from MultiPeak and changed only names I think.

iljah commented 8 years ago

Looks like beginning of the file is messed up, opening tag of BLOCKIDS is only partially there:

hexdump -C -n 100 initial-grid.0000000.vlsv 
00000000  3c 56 4c 53 56 3e 0a 20  00 00 00 00 00 00 00 00  |<VLSV>. ........|
00000010  49 44 53 20 61 72 72 61  79 73 69 7a 65 3d 22 31  |IDS arraysize="1|
00000020  36 30 22 20 64 61 74 61  73 69 7a 65 3d 22 34 22  |60" datasize="4"|
00000030  20 64 61 74 61 74 79 70  65 3d 22 75 69 6e 74 22  | datatype="uint"|
00000040  20 6d 65 73 68 3d 22 53  70 61 74 69 61 6c 47 72  | mesh="SpatialGr|
00000050  69 64 22 20 6e 61 6d 65  3d 22 61 76 67 73 22 20  |id" name="avgs" |
00000060  76 65 63 74                                       |vect|

ykempf commented 8 years ago

I am pretty sure it's not a matter of project. Check that https://github.com/fmihpc/vlsv/blob/master/vlsv_writer.cpp#L126 https://github.com/fmihpc/vlsv/blob/master/vlsv_writer.cpp#L139 and https://github.com/fmihpc/vlsv/blob/master/vlsv_writer.cpp#L140 do what they should, as https://github.com/fmihpc/vlsv/blob/master/vlsv_writer.cpp#L151 then writes the footer in that position.

iljah commented 8 years ago

This

        // myrank == master
         if (dryRunning == false) {
            MPI_File_seek(fileptr,0,MPI_SEEK_END);
            MPI_File_get_position(fileptr,&viewOffset);
            MPI_File_get_byte_offset(fileptr,viewOffset,&endOffset);
         }
        std::cout << "viewOffset: " << viewOffset << ", endOffset: " << endOffset << std::endl;

gives

viewOffset: 43320, endOffset: 0
viewOffset: 37080, endOffset: 0

ykempf commented 8 years ago

Well and this is where you should have a value corresponding to your file size as endOffset.

iljah commented 8 years ago

This test program behaves identically, I'll check if my openmpi installation is messed up:

#include "cstdlib"
#include "iostream"
#include "mpi.h"
int main(int argc, char* argv[]) {
    MPI_Init(&argc, &argv);
    MPI_File f;
    if (MPI_File_open(MPI_COMM_WORLD, "test", MPI_MODE_RDONLY, MPI_INFO_NULL, &f) != MPI_SUCCESS) {
        abort();
    }
    if (MPI_File_seek(f, 0, MPI_SEEK_END) != MPI_SUCCESS) {
        abort();
    }
    MPI_Offset offset1;
    if (MPI_File_get_position(f, &offset1) != MPI_SUCCESS) {
        abort();
    }
    std::cout << "offset1: " << offset1 << std::endl;
    MPI_Offset offset2;
    if (MPI_File_get_byte_offset(f, offset1, &offset2) != MPI_SUCCESS) {
        abort();
    }
    std::cout << "offset2: " << offset2 << std::endl;
    MPI_Finalize();
}

iljah commented 8 years ago

Looks like my openmpi installation was broken. I reinstalled better and now vlsv seems to write files correctly.

fmihpc / vlasiator

Footer is header in vlasiator output #212