Some variables in vtkMultiBlockDataSet are not available

[cssherman]

Describe the bug

I am attempting to visualize multiblock vtk files produced by GEOSX in VisIt. These regions include different sets of variables, some of which have the same name across all region. When opened, the variable list in VisIt reflects the first region listed in the .vtm file.

For variables with shared names across regions, I can use the group selection to visualize the model as expected:

If I attempt to do the same for a variable that only exists in one region, I get the following error message: An invalid variable (elementAperture) was specified. If only the appropriate regions are active in group selection, then the visualization works correctly.

Helpful additional information

• Here is the associated GEOSX issue: https://github.com/GEOSX/GEOSX/issues/1402

To Reproduce

To obtain the error message:

1. Modify one of the .vtm files, so that SurfaceElementRegion is listed first
2. Attempt to create a psuedocolor plot of elementAperture

Expected behavior

Each region in a multiblock vtk file should be considered for the variable list, and the code should not produce an error if a variable is missing in a region (or at least provide a more helpful error message).

Attachments

I have attached a small test database and session file here:

geosx_multiregion_example.zip

Desktop

• TOSS (Pascal)
• VisIt Version: 3.1.4

[biagas]

Thank you for providing sample database, I will try it out and see if I can discover the issue.

[cssherman]

@biagas - Thanks! Please let me know if you need anything else

[biagas]

So you are correct, VisIt's VTK reader only reads the first file in a multi-file setting to determine the variable list.

This was a design choice due to the fact that the entire file must be read just to get the meta-information like variable names. This can be quite costly if there are hundreds or thousands of blocks, or even if there are few blocks, but each are rather large files.

A case could be made for allowing the user to accept this cost, perhaps a read option to the effect 'read all blocks at open.'

I see that your 'regions' (VisIt's reader set them up as 'groups' in VisIt's SIL parlance), each have distinct variable lists. I am not sure VisIt would play nice with this setup, even if the full variable list had been read in. Would have to muck with the reader to try it out and see what happens. Can report back on this later.

[biagas]

I performed a quick mod of the VTK reader so that it would read in multiple blocks and do a better job of populating the variable list. As I suspected, VisIt isn't reacting well to not having all the variables defined in each of the individual files. I think the variables would need to be grouped by their respective regions. For vars that aren't present in both regions, I needed to do a SIL selection to choose the correct Region in order for VisIt to plot it correctly.

Do you consider each region to be a different mesh, or computational space? If VisIt separated them and you wanted to plot 'pressure' you would have to choose which mesh/region, and create 2 plots if you wanted to see both, but vars contained in a single region would plot successfully without needing to use the SIL selection.

[cssherman]

GEOSX considers regions to be areas with a set of constitutive models. In practice this could look like:

1) A solid material with a linear-elastic model, darcy flow model 2) A solid material with a linear-elastic model, impermeable to flow 3) A surface region where contact is evaluated, parallel-plate flow model (representing a propagating fracture)

Here, (1) and (2) could exist in the same solid mesh, and (3) would correspond to a separate surface mesh. (1), (2), and (3) could also have distinct variables. I should also mention that we have an active PR to enable multi-mesh models and vtk outputs. Hopefully this makes sense... We could also bring someone like Randy Settgast into the conversation in case we need to address our vtk format.

In practice, a typical non-super-user would likely only run a model with a single solid and single surface region. With our legacy code + silo output, we typically required the user to generate two plots to track properties of the solid/surface regions.

[cssherman]

I should also mention that my current work-around is a python script that breaks the vtm file up so that the regions are in different files, and then use the database correlation feature to keep everything in sync.

[biagas]

I think your use case really needs multi-mesh functionality, not currently possible with VTK files (that I know of). That said, the portion of the VTK reader in VisIt that parses '.vtm' files could be modified to allow different conventions than currently supported:

1. Add a read-option that would allow the highest level blocks to be treated as separate meshes. The highest level blocks are currently treated as 'groups' in VisIt.
2. Update the .vtm parser that VisIt uses to support nested vtkMultiBlockDataSet at the root level, and treat each vtkMultiBlockDataSet as a separate mesh, the .vtm file would look like this:

<?xml version="1.0"?>
<VTKFile type="vtkMultiBlockDataSet" version="1.0">
    <vtkMultiBlockDataSet>
        <Block name="CellElementRegion">
            <Block name="Domain">
                <DataSet name="rank_0" file="000000/0_Domain.vtu" />
            </Block>
        </Block>
        <Block name="WellElementRegion" />
    </vtkMultiBlockDataSet>
    <vtkMultiBlockDataSet>
        <Block name="SurfaceElementRegion">
            <Block name="Fracture">
                <DataSet name="rank_0" file="000000/0_Fracture.vtu" />
            </Block>
        </Block>
    </vtkMultiBlockDataSet>
</VTKFile>

Of course, if changing the format of your output files isn't feasible, then option 1 would be the only choice.

@brugger1, @markcmiller86 Would you have other suggestions?

[cssherman]

Thanks for your help on this - I'm planning to check in with other GEOSX devs this week to get their thoughts on this issue.

[povolny1]

It may be worth noting that we're planning on adopting the following hierarchy for multibody problems (PR on the GEOSX side: https://github.com/GEOSX/GEOSX/pull/1742):

<?xml version="1.0"?>
<VTKFile type="vtkMultiBlockDataSet" version="1.0">
    <vtkMultiBlockDataSet>
        <Block name="backgroundGrid">
            <Block name="CellElementRegion">
                <Block name="Region1">
                    <DataSet name="rank_0" file="0.000000/0_Region1.vtu" />
                    <DataSet name="rank_1" file="0.000000/1_Region1.vtu" />
                </Block>
            </Block>
            <Block name="WellElementRegion" />
            <Block name="SurfaceElementRegion" />
        </Block>
        <Block name="particles">
            <Block name="CellElementRegion">
                <Block name="Region2">
                    <DataSet name="rank_0" file="0.000000/0_Region2.vtu" />
                    <DataSet name="rank_1" file="0.000000/1_Region2.vtu" />
                </Block>
            </Block>
            <Block name="WellElementRegion" />
            <Block name="SurfaceElementRegion" />
        </Block>
    </vtkMultiBlockDataSet>
</VTKFile>

[biagas]

I am working of modifying our VTK reader to accommodate your use-case.

Can it be assumed that each dataset within a given region will have the same variables? If yes, the up-front cost of finding all the variable names can be reduced.

@povolny1 in your example, should the 'backgroundGrid' and 'particles' blocks be considered separate meshes?

[povolny1]

@biagas Apologies for the late reply - yes, ideally we'd like to be able to simultaneously plot independent fields from 'backgroundGrid' and 'particles.'

Also, all the elements in a region use the same constitutive model, so they'd all have the same fields/variables.

[biagas]

@povolny1, any progress on this front in terms of solidifying the hierarchical structure GEOSX will be using for .vtm files?

I've pulled something together which I believe addresses most of the issues raised. I hacked the previously provided sample data to match the proposed hierarchy for testing, but 'real' data would be better.

Trying to get this in for our 3.3 release. It may be disabled by default (I view it as experimental at this point without good data for testing).

If your .vtm file structure is still a moving target, I can put this on hold and add it into VisIt when your team is ready.

[povolny1]

I'm hoping to get the .vtm hierarchy finalized soon. @biagas What would be the best way to go about testing your changes? Should I just send you a dataset?

[biagas]

To start with, sending me a dataset would be best. I had something sort of working as a separate plugin. But users would have to then select 'GEOSX' from the 'open as' pulldown, so I am currently trying to integrate it with the current VTK plugin, to save user's a step. It also reduces code duplicaton.

Once I think I have things working, we can talk about sharing the plugin with you for your testing before I make a final commit in VisIt.

[biagas]

@cssherman , @pvolny1 what is the status of this on your end? I have been waiting to hear back on the final specs for your .vtm files.

[cssherman]

@biagas - sorry for dropping the ball on this! I will put together some example outputs that are produced by GEOS, so that we can evaluate where we are on this problem.