DA fails while adding material

[stvdwtt]

(Going off of the branch in #219)

There's some issue in how the number of dofs is tracked when adding material with data assimilation. I'll look at this by early next week. When I address this, I plan to add an integration test with DA + added material.

[stvdwtt]

I've located the issue. The problem actually happens for all ensemble runs. ThermalPhysics::add_material only adds new DOFs where the fe_index was nonzero and then sets fe_index to zero. The issue is that after the first ensemble member adds material, the fe_index is already switched so all other ensemble members don't add any DOFs there -- they see the fe_index and think that no more material needs to be added.

Fixing this is somewhat difficult because I think the fe_index needs to be changed to transfer the solution. I don't think I can simply delay to flip fe_index until going through all ensemble members for this reason. One option might be to pull it out of ThermalPhysics so that I can do the whole ensemble together. That would cause somewhat significant changes.

I'm still trying to come up with alternatives that can be implemented with a light touch.

[Rombur]

The issue is that after the first ensemble member adds material, the fe_index is already switched so all other ensemble members don't add any DOFs there -- they see the fe_index and think that no more material needs to be added.

How is it possible? The fe_index is attached to a DoFHandler and each ensemble member has its own DoFHandler. You changing one DoFHandler internal data (the fe_index) has not effect on the other DoFHandler.

I think that the problem comes from https://github.com/adamantine-sim/adamantine/blob/9aa03933f18f87e98ffbf82b3f115d579f84d016/application/adamantine.hh#L1605-L1607 and https://github.com/adamantine-sim/adamantine/blob/9aa03933f18f87e98ffbf82b3f115d579f84d016/application/adamantine.hh#L1652-L1654 Instead of using the ensemble member 0 all the time to add material, you need to get elements to activate for each ensemble members. Right now, you get cells to activate from ensemble member 0 which obviously don't exist in the other ensemble members and so no cell is activated. Using ensemble member 0 all the time makes sense geometrically because it's the same geometric cells but we are using dealii::DoFHandler<dim>::active_cell_iterator and so the values are different for each ensemble member.

[stvdwtt]

@Rombur, thanks -- that makes sense now. I made those changes and it's no longer crashing. I'm working on a test now to demonstrate correctness.

[stvdwtt]

Fix in #224