gemini-hlsw / lucuma-core

Lucuma Core Classes
Other
9 stars 9 forks source link

sky calculations flaky test #463

Open swalker2m opened 2 years ago

swalker2m commented 2 years ago
==> X lucuma.core.math.skycalc.ImprovedSkyCalcSuiteJVM.Arbitrary sky calculations  0.077s munit.FailException: /home/runner/work/lucuma-core/lucuma-core/modules/tests/jvm/src/test/scala/lucuma/core/math/skycalc/ImprovedSkyCalcSuiteJVM.scala:61 assertion failed
2330
60:          assert((results.altitudeRaw +- 1e-12).isWithin(javaCalc.getAltitude))
2331
61:          assert((results.azimuthRaw +- 1e-12).isWithin(javaCalc.getAzimuth))
2332
62:          assert((results.parallacticAngleRaw +- 1e-12).isWithin(javaCalc.getParallacticAngle))
2333

2334
Failing seed: TIF8MuF3zIk5ELXVVIklHqPmo2uKLvN9rMllM19aEOK=
2335
You can reproduce this failure by adding the following override to your suite:
2336

2337
  override val scalaCheckInitialSeed = "TIF8MuF3zIk5ELXVVIklHqPmo2uKLvN9rMllM19aEOK="
2338

2339
Falsified after 18 passed tests.
2340
> ARG_0: Place(Dec(+31:59:05.224193),Angle.fromDMS(77,39,1,601,664),Quantity(0),Pacific/Norfolk)
2341
> ARG_1: 1901-01-01T00:00Z
2342
> ARG_2: Coordinates(00:00:00.000000 -31:59:05.224192)
2343
    at munit.FailException.copy(FailException.scala:27)
2344
    at munit.FailException.withMessage(FailException.scala:20)
2345
    at munit.FailException.withMessage(FailException.scala:3)
2346
    at munit.ScalaCheckSuite.propToTry(ScalaCheckSuite.scala:91)
2347
    at munit.ScalaCheckSuite.$anonfun$scalaCheckPropTransform$3(ScalaCheckSuite.scala:46)
2348
    at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:475)
2349
    at munit.Suite$$anon$1.execute(Suite.scala:26)
2350
    at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
2351
    at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
2352
    at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
2353
    at scala.concurrent.impl.Promise$DefaultPromise.transform(Promise.scala:128)
2354
    at munit.internal.FutureCompat$ExtensionFuture.transformCompat(FutureCompat.scala:16)
2355
    at munit.ScalaCheckSuite.$anonfun$scalaCheckPropTransform$2(ScalaCheckSuite.scala:48)
2356
    at munit.GenericTest.$anonfun$withBodyMap$1(GenericTest.scala:33)
2357
    at munit.MUnitRunner.$anonfun$runTestBody$1(MUnitRunner.scala:296)
swalker2m commented 1 year ago
==> X lucuma.core.math.skycalc.ImprovedSkyCalcSuiteJVM.Arbitrary sky calculations  0.057s munit.ComparisonFailException: /Users/swalker/dev/lucuma-core/modules/tests/jvm/src/test/scala/lucuma/core/math/skycalc/ImprovedSkyCalcSuiteJVM.scala:61 values are not the same expected: 257.63675620106716 but was: 257.63675620106835
60:          assertEqualsDouble(results.altitudeRaw, javaCalc.getAltitude,  1e-12)
61:          assertEqualsDouble(results.azimuthRaw, javaCalc.getAzimuth, 1e-12)
62:          assertEqualsDouble(results.parallacticAngleRaw, javaCalc.getParallacticAngle, 1e-12)

Failing seed: Bd1A3jFrhQSwtrGHQgBaI1riv8SHU47QJjIAnJeGjxM=
You can reproduce this failure by adding the following override to your suite:

  override val scalaCheckInitialSeed = "Bd1A3jFrhQSwtrGHQgBaI1riv8SHU47QJjIAnJeGjxM="

Falsified after 25 passed tests.
> ARG_0: Place(Dec(-31:59:05.224192),763145224192,1,America/Araguaina)
> ARG_1: 1901-01-01T00:00Z
> ARG_2: Coordinates(20:45:44.102192 -31:59:05.224192)
hugo-vrijswijk commented 1 year ago

Changing the delta on line 62 and 63 to 1e-11 instead of 1e-12 fixes the issue. I'm not sure what the precision is we should have, but it seems that either we don't handle the needed precision properly, or the test delta is too high

swalker2m commented 6 months ago
==> X lucuma.core.math.skycalc.ImprovedSkyCalcSuiteJVM.Arbitrary sky calculations  0.027s munit.ComparisonFailException: /Users/swalker/dev/lucuma-core/modules/tests/jvm/src/test/scala/lucuma/core/math/skycalc/ImprovedSkyCalcSuiteJVM.scala:61 values are not the same expected: 253.83892822548432 but was: 253.83892822548603
60:          assertEqualsDouble(results.altitudeRaw, javaCalc.getAltitude,  1e-12)
61:          assertEqualsDouble(results.azimuthRaw, javaCalc.getAzimuth, 1e-12)
62:          assertEqualsDouble(results.parallacticAngleRaw, javaCalc.getParallacticAngle, 1e-12)

Failing seed: 5ubD978aA4xHy4r0rsjeSPIVEV0JSMFzyGn2EnXWZtM=
You can reproduce this failure by adding the following override to your suite:

  override val scalaCheckInitialSeed = "5ubD978aA4xHy4r0rsjeSPIVEV0JSMFzyGn2EnXWZtM="

Falsified after 5 passed tests.
> ARG_0: Place(Dec(+00:00:00.000000),938482303035,1607,America/Indiana/Vincennes)
> ARG_1: 1901-01-01T00:00Z
> ARG_2: Coordinates(00:00:00.000000 +00:00:00.000000)