search

nightscout / Trio

MIT License
49 stars 154 forks source link

Submodule tests are not run automatically, failure in CalibrationsTest #146

Open marionbarker opened 2 months ago

marionbarker commented 2 months ago

Describe the bug

  1. When running Tests in Xcode, expect submodule tests to run automatically
  2. The CalibrationsTests in FreeAPSTest is failing and should be fixed

To Reproduce Steps to reproduce the behavior:

  1. Open Xcode
  2. Build to a simulator
  3. Run Tests

Expected behavior

  1. All tests should run (including submodules.
  2. All tests should pass

The 4 XCTAssert line that fail are: lines 34, 36, 38 and 42 for FreeAPSTests/CalibrationsTests.swift

marionbarker commented 2 months ago

Got some feedback from @itsmojo.

Opinion:

This is not a blocking issue, so can deal with this later - just capturing the information here.

Feedback from Joe:

the issue seems to be that the expected values are only returned on the first Test run for a given iPhone simulator.

After the first run, the values seem to be a consistent values of 0.9500000000000307, 15.999999999996717, 91.99999999999918 & 2 (instead of the 0.8, 37, 101 & 1 in the initial run that are being tested for).

So this seems like this is a stateful test which really should be reworked to have consistent results on each run.

Also, using XCTAssertTrue() for these tests is not as helpful as using XCTAssertEqual() so that you can see both values on a failure and so an optional accuracy parameter can be used to deal with non exact floating point values like 15.999999999996717.

Example using XCTAssertEqual()

XCTAssertEqual(calibrationService.slope, 0.95, accuracy: 0.0001)

github-actions[bot] commented 1 week ago

hey 👋 - silence for 30 days 🤐 ... anybody? triage is required!