Overwrites of objects in Silo files have always been a bit iffy/kludgey.
On the PDB driver, they work as long as what is overwritten as no bigger than what is already in the file. PDB driver literally writes the new data right over the old data and if the new data is bigger, it will corrupt other stuff in the file in sometimes not easy to diagnose ways.
On the HDF5 driver, the way things are currently designed, we orphan the old data in the file and metadata is updated to point to newly written data.
In any event, when we overwrite data to existing Silo files from python module, python only understands double precision data, not 32-bit float data. So, if read an array into python interp, manipulate it and (over)write it back out, you will wind up corrupting the file.
The solution is for python C extension lib to detect this case and convert back to float.
markcmiller86
commented
2 months ago
Overwrites of objects in Silo files have always been a bit iffy/kludgey.
On the PDB driver, they work as long as what is overwritten as no bigger than what is already in the file. PDB driver literally writes the new data right over the old data and if the new data is bigger, it will corrupt other stuff in the file in sometimes not easy to diagnose ways.
On the HDF5 driver, the way things are currently designed, we orphan the old data in the file and metadata is updated to point to newly written data.
In any event, when we overwrite data to existing Silo files from python module, python only understands double precision data, not 32-bit float data. So, if read an array into python interp, manipulate it and (over)write it back out, you will wind up corrupting the file.
The solution is for python C extension lib to detect this case and convert back to float.