search

georust / proj

Rust bindings for the latest stable release of PROJ
https://docs.rs/proj
Apache License 2.0
143 stars 46 forks source link

Intermittent terrifying CI failures #55

Closed michaelkirk closed 3 years ago

michaelkirk commented 3 years ago

We've seen these off and on for a while. I wanted to start collecting data on them to see if we can track down what's going on.

this time anyway it was a macos failure. When we do fail in this way, is it always a mac?

Note this is not the same issue as #52, which is very reproducible if you're on an arm64 mac.

running 9 tests
test src/lib.rs - (line 31) ... ok
test src/lib.rs - (line 56) ... ok
test src/lib.rs - (line 162) ... ok
test src/lib.rs - (line 79) ... ok
test src/proj.rs - proj::Proj::convert (line 641) ... ok
test src/proj.rs - proj::Proj::convert_array (line 697) ... ok
test src/proj.rs - proj::Proj::new_known_crs (line 469) ... ok
test src/proj.rs - proj::Proj::project_array (line 730) ... FAILED
test src/proj.rs - proj::ProjBuilder::proj_known_crs (line 391) ... ok

failures:
---- src/proj.rs - proj::Proj::project_array (line 730) stdout ----
Test executable failed (exit code 101).

stderr:
thread 'main' panicked at 'assert_relative_eq!(v[0].x(), 500119.70, epsilon = 1e-2)

    left  = -2211720633.1788964
    right = 500119.7

Here's a link to the most recent intermittent failure: https://github.com/georust/proj/runs/1653291398?check_suite_focus=true

This is a terribly confusing example to link to because there are other legitimate failures for that job as I worked through the fallout of integrating approx. This issue is just about the "proj-sys macos" build showing the intermittent failure above, the maybe it sometimes affects other platforms too? unclear...

urschrei commented 3 years ago

I'm on Mac too, so this will be slightly annoying to chase down. The fact that it's occurring in the array method is what makes me nervous (can't recall whether it's always the same test that fails) because that seems more likely to be clobbering memory somewhere, even though the Rust side isn't doing anything particularly complex.

urschrei commented 3 years ago

Some more digging: on master, testing test_array_convert alone with cargo test array passes every time (n=20). Running cargo test causes it to fail every time (On stable 1.49.0, intel macOS Big Sur).

urschrei commented 3 years ago

It's also not related to the specific CRS or coordinates; I can reproduce the above pass / fail using different ones.

michaelkirk commented 3 years ago

👻 😱

urschrei commented 3 years ago

See #56 for some progress.