writeNative does not write frozen fields

[jacobmerson]

It turns our that if fields are frozen, then the field will not write out to an smb file.

[cwsmith]

Is support for writing frozen fields needed?

[jacobmerson]

In short yes. It would be useful for my applications.

I use frozen fields to put my data in a format that I can easily transfer to an accelerator as needed. This is nice because it allows me to avoid a copy into fields by directly accessing the array based structures.

It turns out that dumping out a VTK file does the correct thing and can write the frozen fields. I am currently dumping out a mesh (smb file) and some metadata as a way of restarting my analysis, so the VTK format won't work for me.

If I unfreeze the fields and then refreeze them, this a) allocates and deallocates the arrays which may be expensive, and b) invalidates any pointers that I may have to that data requiring that I reset those pointers (which may not be readily accessible) every time I checkpoint.

[cwsmith]

How about manually creating an array for each field and leaving the pumi fields un-frozen? If the mesh and field are static then it sounds like this could work.

[jacobmerson]

I can do this, however I will have a higher memory overhead, and I will have to copy the data from the array to the tag field every time I want to write to disk.

Certainly this will be the approach to take if we decide that we don't want to fully support frozen fields. If the decision is to not support frozen fields everywhere we need some documentation.

[cwsmith]

I will have to copy the data from the array to the tag field every time I want to write to disk

I'm a bit confused about the purpose of a frozen field. I thought they were supposed to be read-only?

[jacobmerson]

That might be the original intended purpose (which would make the name make more sense), however, this is not indicated anywhere in the documentation and when you unfreeze a field it copies the data back into the mesh.

[jacobmerson]

Looking through the API's it seems as though the intention of a frozen field is not to be read only. But this leaves us with the problem that any time we unfreeze a field, we need to re-freeze it, and get the new data array pointer.

For the specific issue mentioned here, we may want to add a call to: https://github.com/SCOREC/core/blob/b8e5326041be72a56d2a1f4d0606e06ffe1c4b22/apf/apfMesh2.h#L68

In the write native function.

[cwsmith]

There is another catch here; before mesh data arrays are written to .smb they must be 'compacted' via reordering if there were topological (i.e., mesh adapt) or distribution changes (i.e., load balancing/partitioning).

Do you know what happens if meshAdapt, reordering, or load balancing/partitioning is called when frozen fields exist? Do those operations succeed? If so, what happens to the underlying frozen fields?

I'm thinking the API would be safer/clearer if it were getFieldArray(...) and setFieldArray(...) instead of freeze/unfreeze. I'm not sure the memory savings are worth the book keeping.