crash when doing mesh adaptivity + LineMaterialRealSampler

[jessecarterMOOSE]

Bug Description

MOOSE crashes when using both mesh adaptivity and LineMaterialRealSampler VectorPostprocessor using a recent master commit (https://github.com/idaholab/moose/tree/e52989795).

Steps to Reproduce

Simply add the following lines to something like test/tests/adaptivity/cycles_per_step/cycles_per_step.i:

[Materials]
  [./const]
    type = GenericConstantMaterial
    prop_names = 'matA'
    prop_values = '1.0'
  [../]
[]

[VectorPostprocessors]
  [./mats]
    type = LineMaterialRealSampler
    property = 'matA'
    start = '0 0 0'
    end = '1 1 0'
    sort_by = 'x'
  [../]
[]

Impact

Prevents using these features together.

[jessecarterMOOSE]

@lindsayad @fdkong I think I ran into this when testing out #15338, but now I get it without any automatic scaling.

A full input file that produces this looks like:

[Mesh]
  type = GeneratedMesh
  dim = 2
  nx = 10
  ny = 10
[]

[Variables]
  [u]
  []
[]

[Kernels]
  [diff]
    type = CoefDiffusion
    variable = u
    coef = 0.1
  []
  [time]
    type = TimeDerivative
    variable = u
  []
[]

[BCs]
  [left]
    type = DirichletBC
    variable = u
    boundary = left
    value = 0
  []
  [right]
    type = DirichletBC
    variable = u
    boundary = right
    value = 1
  []
[]

[Materials]
  [./const]
    type = GenericConstantMaterial
    prop_names = 'matA'
    prop_values = '1.0'
  [../]
[]

[VectorPostprocessors]
  [./mats]
    type = LineMaterialRealSampler
    property = 'matA'
    start = '0 0 0'
    end = '1 1 0'
    sort_by = 'x'
  [../]
[]

[Executioner]
  type = Transient
  num_steps = 5
  dt = 0.1
  solve_type = PJFNK
  petsc_options_iname = '-pc_type -pc_hypre_type'
  petsc_options_value = 'hypre boomeramg'
[]

[Adaptivity]
  initial_steps = 2
  cycles_per_step = 2
  marker = marker
  initial_marker = marker
  max_h_level = 2
  [Indicators/indicator]
    type = GradientJumpIndicator
    variable = u
  []
  [Markers/marker]
    type = ErrorFractionMarker
    indicator = indicator
    coarsen = 0.1
    refine = 0.7
  []
[]

[Outputs]
  exodus = true
[]

[YaqiWang]

Could be related with https://github.com/idaholab/moose/issues/12414 and hopefully the new implementation will be soon with the ray-tracing module checked in.

[jessecarterMOOSE]

Actually playing around a bit more I get a crash without adaptivity at those particular start/end points on the vectorpostprocessor. If I move the points a bit so they lie on the interior and not the edge (start=(0.1,0.1,0), end=(0.9,0.9,0)), then turn on mesh adaptivity, I get a few more timesteps before getting this in debug mode:

No index 4294967295 in ghosted vector.
Vector contains [0,2060)
And empty ghost array.

And then a big stack trace:

#0  0x00007fffdaa374b9 in waitpid () from /lib64/libc.so.6
#1  0x00007fffda9b4f62 in do_system () from /lib64/libc.so.6
#2  0x00007fffda9b5311 in system () from /lib64/libc.so.6
#3  0x00007fffe28649c4 in (anonymous namespace)::gdb_backtrace (out_stream=...) at /home/cartjess/projects/moose/scripts/../libmesh/src/base/print_trace.C:162
#4  0x00007fffe2864d50 in libMesh::print_trace (out_stream=...) at /home/cartjess/projects/moose/scripts/../libmesh/src/base/print_trace.C:209
#5  0x00007fffe2858a37 in libMesh::MacroFunctions::report_error (file=0x7fffea547370 "/home/cartjess/projects/moose/scripts/../libmesh/installed/include/libmesh/petsc_vector.h", line=1059, date=0x7fffea545132 "Nov 13 2020", time=0x7fffea545129 "16:56:25") at /home/cartjess/projects/moose/scripts/../libmesh/src/base/libmesh_common.C:85
#6  0x00007fffe9baab6b in libMesh::PetscVector<double>::map_global_to_local_index (this=0xd16b80, i=4294967295) at /home/cartjess/projects/moose/scripts/../libmesh/installed/include/libmesh/petsc_vector.h:1059
#7  0x00007fffe9ba91b4 in libMesh::PetscVector<double>::operator() (this=0xd16b80, i=4294967295) at /home/cartjess/projects/moose/scripts/../libmesh/installed/include/libmesh/petsc_vector.h:1074
#8  0x00007fffe8eeb604 in MooseVariableData<double>::computeMonomialValues (this=0xd6b210) at /home/cartjess/projects/moose/framework/src/variables/MooseVariableData.C:1164
#9  0x00007fffe8eda721 in MooseVariableConstMonomial::computeElemValues (this=0xd6ae30) at /home/cartjess/projects/moose/framework/src/variables/MooseVariableConstMonomial.C:46
#10 0x00007fffe8c98fc1 in AuxiliarySystem::reinitElem (this=0xd14b90, tid=0) at /home/cartjess/projects/moose/framework/src/systems/AuxiliarySystem.C:351
#11 0x00007fffe933782a in FEProblemBase::reinitElem (this=0xce06f0, elem=0xcb7a60, tid=0) at /home/cartjess/projects/moose/framework/src/problems/FEProblemBase.C:1722
#12 0x00007fffe9ab0dd6 in LineMaterialSamplerBase<double>::execute (this=0xe635d0) at /home/cartjess/projects/moose/framework/build/header_symlinks/LineMaterialSamplerBase.h:176
#13 0x00007fffe934901b in FEProblemBase::joinAndFinalize(TheWarehouse::QueryCache<>, bool) (this=0xce06f0, query=..., isgen=true) at /home/cartjess/projects/moose/framework/src/problems/FEProblemBase.C:3709
#14 0x00007fffe934a851 in FEProblemBase::computeUserObjectsInternal(MooseEnumItem const&, Moose::AuxGroup const&, TheWarehouse::QueryCache<>&) (this=0xce06f0, type=..., group=@0x7fffffffaed4: Moose::POST_AUX, query=...) at /home/cartjess/projects/moose/framework/src/problems/FEProblemBase.C:3870
#15 0x00007fffe934947e in FEProblemBase::computeUserObjects (this=0xce06f0, type=..., group=@0x7fffffffaed4: Moose::POST_AUX) at /home/cartjess/projects/moose/framework/src/problems/FEProblemBase.C:3753
#16 0x00007fffe9348c82 in FEProblemBase::execute (this=0xce06f0, exec_type=...) at /home/cartjess/projects/moose/framework/src/problems/FEProblemBase.C:3660
#17 0x00007fffe9a37927 in PicardSolve::solveStep (this=0xd55c10, begin_norm_old=1.7976931348623157e+308, begin_norm=@0xee0160: 0, end_norm_old=1.7976931348623157e+308, end_norm=@0x1065400: 0, relax=false, relaxed_dofs=...) at /home/cartjess/projects/moose/framework/src/executioners/PicardSolve.C:487
#18 0x00007fffe9a35f80 in PicardSolve::solve (this=0xd55c10) at /home/cartjess/projects/moose/framework/src/executioners/PicardSolve.C:236
#19 0x00007fffe9250c26 in TimeStepper::step (this=0xed1940) at /home/cartjess/projects/moose/framework/src/timesteppers/TimeStepper.C:163
#20 0x00007fffe9a3d904 in Transient::takeStep (this=0xd55950, input_dt=-1) at /home/cartjess/projects/moose/framework/src/executioners/Transient.C:431
#21 0x00007fffe9a3d015 in Transient::execute (this=0xd55950) at /home/cartjess/projects/moose/framework/src/executioners/Transient.C:314
#22 0x00007fffe9d74570 in MooseApp::executeExecutioner (this=0xa2d2e0) at /home/cartjess/projects/moose/framework/src/base/MooseApp.C:939
#23 0x00007fffe9d74e8c in MooseApp::run (this=0xa2d2e0) at /home/cartjess/projects/moose/framework/src/base/MooseApp.C:1066
#24 0x0000000000413de1 in main (argc=3, argv=0x7fffffffbd58) at /home/cartjess/projects/moose/test/src/main.C:33
[Inferior 1 (process 12329) detached]
[0] /home/cartjess/projects/moose/scripts/../libmesh/installed/include/libmesh/petsc_vector.h, line 1059, compiled Nov 13 2020 at 16:56:25
[unset]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1
:
system msg for write_line failure : Bad file descriptor
[sawtooth2:mpi_rank_0][MPIDI_CH3_Abort] application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0: Bad file descriptor (9)

[loganharbour]

Yeah, you're hitting some bad, literal corner cases there it seems.

I'll be sure to port the sampling capability over from ray tracing when #16029 gets in. We've handled the corner cases much better there through lots of trial and error. That should be able to be done by the end of the year.

Is this something that has a time constraint for you @jessecarterMOOSE?

[jessecarterMOOSE]

@loganharbour No time constraint. Just more of a minor inconvenience. I can certainly make do for the time being.