search

visit-dav / visit

VisIt - Visualization and Data Analysis for Mesh-based Scientific Data
https://visit.llnl.gov
BSD 3-Clause "New" or "Revised" License
428 stars 111 forks source link

Re-code Global element id based ghost zone comm to use 64 bit unsigned int type #3439

Open markcmiller86 opened 5 years ago

markcmiller86 commented 5 years ago

Describe the bug

For UCD meshes, VisIt supports communication of ghost elements (nodes and/or zones) using global ids. However, that logic appears to assume those are int type using vtkIntArray types. There is no type checking either.

For large meshes, 32 bits are not sufficient to store global node ids. So, a number of file formats, Silo and Exodus for sure, use 64-bit types.

We have an example from Exodus which fails. I dunno why we haven't seen this before with Silo except that perhaps all examples of UCD meshes in Silo are also storing pre-computed ghosts.

VisIt calls GetAuxiliaryData on the plugin with the GLOBAL_NODE_IDS tag. The plugin responds with a vtkDataArray of type of its choosing and VisIt adds the name "avtGlobalNodeIds" to the array. Later in the ghost zone comm. logic, it queries for the array named "avtGlobalNodeIds" and assumes it is getting a vtkIntArray back.

I rated impact high and likelihood low because it isn't very common.

There is a broader memory question here. Should global element id based ghost zone comm be converted to 64 bit types exclusively? That would represent a 2x memory hit in the global id data for smaller cases where it isn't necessary?

However, if we take into account that connectivity data, coordinates and a single scalar field are typically needed to produce any useful visualization and assume hexahedral connectivities, that all represents 12 problem-sized arrays of 32 bit quantities. In that context, the additional cost for always assuming 64-bit values for global element ids represents less than a 10% memory hit (1 additional problem-sized, 32 bit array among 13).

So, I conclude we should just convert all the global element id logic there to use 64-bit unsigned quantities and all plugins should be required to serve up 64 bit unsigned quantities for these values. Transform manager can take care of converting plugin data.

To Reproduce

See #3409

When we fix, we should add the data attached to #3409 to our test suite and test with Exodus.

markcmiller86 commented 4 years ago

Putting back into un-reviewed state to be sure we consider this issue again.