some failing tests

[michaelkirk]

We're seeing some failures in CI for https://github.com/georust/proj/pull/98 e.g.

https://github.com/georust/proj/runs/5014199807?check_suite_focus=true

Test failure look unrelated?

Indeed! I'm looking into that.

The failures seem eerily similar to the ones I found in: https://github.com/georust/proj/pull/97#issuecomment-994200364

I opened this PR just as a way to gather more data, to see if I could find a pattern.

CI Run from this branch: https://github.com/georust/proj/runs/5013682778?check_suite_focus=true

• ✅ rust 1.52
• ⭕️ rust 1.53
• ❌ default features

  failures:
  
  ---- proj::test::test_inverse_projection stdout ----
  thread 'proj::test::test_inverse_projection' panicked at 'assert_relative_eq!(t.x(), 0.43633200013698786)
  
      left  = 0.43636512759931073
      right = 0.43633200013698786
  
  ', src/proj.rs:1069:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  
  ---- proj::test::test_london_inverse stdout ----
  thread 'proj::test::test_london_inverse' panicked at 'assert_relative_eq!(t.x(), 0.0023755864830313977)
  
      left  = 0.002407322568220306
      right = 0.0023755864830313977
  ', src/proj.rs:1086:9
  
  ---- proj::test::test_projection stdout ----
  thread 'proj::test::test_projection' panicked at 'assert_relative_eq!(t.x(), 500119.7035366755, epsilon = 1e-5)
      left  = 499972.70746607287
      right = 500119.7035366755
  
  ', src/proj.rs:1054:9
  
  failures:
      proj::test::test_inverse_projection
      proj::test::test_london_inverse
      proj::test::test_projection
  

• ❌ --features=network

  failures:
  
  ---- proj::test::test_inverse_projection stdout ----
  thread 'proj::test::test_inverse_projection' panicked at 'assert_relative_eq!(t.x(), 0.43633200013698786)
  
      left  = 0.43636512759931073
      right = 0.43633200013698786
  
  ', src/proj.rs:1069:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  
  ---- proj::test::test_london_inverse stdout ----
  thread 'proj::test::test_london_inverse' panicked at 'assert_relative_eq!(t.x(), 0.0023755864830313977)
  
      left  = 0.002407322568220306
      right = 0.0023755864830313977
  
  ', src/proj.rs:1086:9
  
  ---- proj::test::test_projection stdout ----
  thread 'proj::test::test_projection' panicked at 'assert_relative_eq!(t.x(), 500119.7035366755, epsilon = 1e-5)
  
      left  = 499972.70746607287
      right = 500119.7035366755
  
  ', src/proj.rs:1054:9
  
  failures:
      proj::test::test_inverse_projection
      proj::test::test_london_inverse
      proj::test::test_projection
  

• ✅ --features=bundled_proj
• ❌ --no-default-features

  failures:
  
  ---- proj::test::test_inverse_projection stdout ----
  thread 'proj::test::test_inverse_projection' panicked at 'assert_relative_eq!(t.x(), 0.43633200013698786)
  
      left  = 0.43636512759931073
      right = 0.43633200013698786
  
  ', src/proj.rs:1069:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  
  ---- proj::test::test_london_inverse stdout ----
  thread 'proj::test::test_london_inverse' panicked at 'assert_relative_eq!(t.x(), 0.0023755864830313977)
  
      left  = 0.002407322568220306
      right = 0.0023755864830313977
  
  ', src/proj.rs:1086:9
  
  ---- proj::test::test_projection stdout ----
  thread 'proj::test::test_projection' panicked at 'assert_relative_eq!(t.x(), 500119.7035366755, epsilon = 1e-5)
  
      left  = 499972.70746607287
      right = 500119.7035366755
  
  ', src/proj.rs:1054:9
  
  failures:
      proj::test::test_inverse_projection
  error: test failed, to rerun pass '--lib'
      proj::test::test_london_inverse
      proj::test::test_projection
  
  test result: FAILED. 12 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.33s
  

• ✅ --features=network bundled_proj
• ❌ --features=network geo-types

  failures:
  
  ---- proj::test::test_inverse_projection stdout ----
  thread 'proj::test::test_inverse_projection' panicked at 'assert_relative_eq!(t.x(), 0.43633200013698786)
  
      left  = 0.43636512759931073
      right = 0.43633200013698786
  
  ', src/proj.rs:1069:9
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  
  ---- proj::test::test_london_inverse stdout ----
  thread 'proj::test::test_london_inverse' panicked at 'assert_relative_eq!(t.x(), 0.0023755864830313977)
  
      left  = 0.002407322568220306
      right = 0.0023755864830313977
  
  ', src/proj.rs:1086:9
  
  ---- proj::test::test_projection stdout ----
  thread 'proj::test::test_projection' panicked at 'assert_relative_eq!(t.x(), 500119.7035366755, epsilon = 1e-5)
  
      left  = 499972.70746607287
      right = 500119.7035366755
  
  ', src/proj.rs:1054:9
  
  failures:
      proj::test::test_inverse_projection
      proj::test::test_london_inverse
      proj::test::test_projection
  
  test result: FAILED. 13 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.34s
  

• ✅ --features=bundled_proj geo-types
• ✅ --features=network bundled_proj geo-types
• ✅ rust 1.54
• ✅ rust 1.55
• ✅ rust 1.56
• ✅ rust 1.57
• ✅ rust 1.58

Curiously in #97, all the rust-1.53 tests were passing and it was the rust-1.52 tests that were failing. I'm not opposed to raising the MSRV, but it doesn't seem like the failures are as simple as "broken on old rust".

[michaelkirk]

I reran with no changes and now see all the rust-1.53 tests as passing, but two from rust-1.54 are now failing... there doesn't seem to be any pattern to the failures that I can see.

CI output

tldr;

• ✅ rust 1.52
• ✅ rust 1.53
• ✅ rust 1.54
  • ✅default features
  • ✅ --features=network
  • ❌ --features=bundled_proj

    failures:
    
    ---- src/lib.rs - (line 56) stdout ----
    Test executable failed (exit code 101).
    
    stderr:
    thread 'main' panicked at 'assert_relative_eq!(result.0, 1450880.29, epsilon = 1e-2)
    
        left  = NaN
        right = 1450880.29
    
    ', src/lib.rs:20:1
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    
    failures:
        src/lib.rs - (line 56)
    

  • ✅ --no-default-features
  • ❌ --features=network bundled_proj

    failures:
    
    ---- src/lib.rs - (line 56) stdout ----
    Test executable failed (exit code 101).
    
    stderr:
    thread 'main' panicked at 'assert_relative_eq!(result.0, 1450880.29, epsilon = 1e-2)
    
        left  = 21993543.96732857
        right = 1450880.29
    
    ', src/lib.rs:20:1
    

  • ✅ --features=network geo-types
  • ✅ --features=bundled_proj geo-types
  • ✅ --features=network bundled_proj geo-types
• ✅ rust 1.55
• ✅ rust 1.56
• ✅ rust 1.57
• ✅ rust 1.58

[lnicola]

I get this one in a doc test on master, but only in release builds:

running 11 tests
test src/lib.rs - (line 56) ... ok
test src/proj.rs - proj::Proj::convert (line 714) ... ok
test src/geo_types.rs - geo_types::geo_types::Coordinate<T> (line 1) ... ok
test src/proj.rs - proj::ProjBuilder::proj_known_crs (line 419) ... ok
test src/lib.rs - (line 31) ... ok
test src/geo_types.rs - geo_types::geo_types::Point<T> (line 28) ... ok
test src/proj.rs - proj::Proj::new_known_crs (line 502) ... ok
test src/lib.rs - (line 179) ... ok
test src/lib.rs - (line 142) ... FAILED
test src/proj.rs - proj::Proj::project_array (line 804) ... ok
test src/proj.rs - proj::Proj::convert_array (line 770) ... ok

failures:

---- src/lib.rs - (line 142) stdout ----
Test executable failed (exit code 101).

stderr:
thread 'main' panicked at 'assert_relative_eq!(result.x(), 158458.67, epsilon = 1e-2)

    left  = -14339492.078978319
    right = 158458.67

[lnicola]

It's strange. The test still fails (with a NaN) as a non-doc test. If I remove the proj_errno_reset, it starts to pass, but test_error_recovery now fails, which I guess invalidates my hopes about functions resetting errno on success.

[lnicola]

Looks like CI passed now :tada:.

[michaelkirk]

Fixed in #104