search

sandialabs / LCM

Laboratory for Computational Mechanics
Other
12 stars 7 forks source link

Failing LCM/ATO tests on Mayer (ARM machine) #8

Closed ikalash closed 3 weeks ago

ikalash commented 6 years ago

The Mayer nightly tests might (fingers crossed...) be stable finally. It looks like some tests are failing there:

http://cdash.sandia.gov/CDash-2-3-0/viewTest.php?onlyfailed&buildid=73330

Schwarz_Cubes | Failed | 3s | Completed (Failed)
ATO:RegHeaviside_3D | Failed | 34s 650ms | Completed (Failed)
ATOT:RegHeaviside_3D | Failed | 42s 560ms | Completed (Failed)
CrystalPlasticity_MinisolverStep_Newton | Failed | 3s 720ms | Completed (Failed)
CrystalPlasticity_MinisolverStep_NewtonLineSearch | Failed | 3s 710ms | Completed (Failed)
CrystalPlasticity_MinisolverStep_TrustRegion | Failed | 3s 470ms | Completed (Failed)
CrystalPlasticity_ThermallyActivatedSlip | Failed | 4s 880ms | Completed (Failed)
CrystalPlasticity_SchwarzBar_modified_np1 | Failed | 3s 60ms | Completed (Failed)

The CrystalPlasticity ones might be just due to the tolerances being so tight (@lxmota , @jwfoulk : can you please check?) . Not sure what is going on with the ATO tests. The Schwarz tests throw the following exception

p=0: *** Caught standard std::exception of type 'std::invalid_argument' :

 /home/projects/albany/nightlyCDashTrilinos/repos/Trilinos/packages/tpetra/core/src/Tpetra_MultiVector_def.hpp:3679:

 Throw number = 1

 Throw test that evaluated to true: (j >= numCols)

 Tpetra::MultiVector<double,int,long long,Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::OpenMP, Kokkos::HostSpace>>::MultiVector(const MultiVector&, const size_t): Input index j (== 1) exceeds valid column index range [0, 1 - 1].
CMake Error at /home/projects/albany/nightlyCDashAlbany/repos/Albany/tests/large/LCM/Schwarz/runtest.cmake:14 (message):
  Albany didn't run: test failed

I'll investigate this when I have time.

bartgol commented 6 years ago

Is this with the current version of master? If so, and you find out it may be related to the branch merged yesterday, let me know, and I'll help.

bartgol commented 5 years ago

@ikalash is this still happening?

ikalash commented 5 years ago

For LCM it has been resolved. I turned off ATO on mayer some time ago, as it didn't seem anyone was interested in resolving the problem and mayer was not a target for ATO. I could try turning it back on temporarily to see what happens, if you like.

bartgol commented 5 years ago

I think we can keep it off. @jrobbin has tests for Albany-Plato on some machine. If he thinks those tests are enough for the ATO team, then we can keep ATO off on mayer.

ikalash commented 5 years ago

Fine by me. Feel free to close the issue. I kept it open in case we wanted to revisit the issue at some point, so that we do not forget.

ikalash commented 3 weeks ago

Don't care anymore about ATO or Mayer. Closing.