search

DeclareDesign / estimatr

estimatr: Fast Estimators for Design-Based Inference
https://declaredesign.org/r/estimatr
Other
131 stars 20 forks source link

Test suite issue on i386 and PowerPC 32-bit for version 1.0.2 #407

Open tillea opened 8 months ago

tillea commented 8 months ago

Hi, when upgrading the Debian package of estimatr from version 1.0.0 to 1.0.2 a test suite error occurs on i386 architecture as you can see in the according test log. It contains

══ Failed tests ════════════════════════════════════════════════════════════════
── Failure ('test-lm-robust-fes.R:589:5'): FEs handle collinear covariates ─────
tidy(lmo)[grepl("cyl", lmo$term), ] not equivalent to tidy(lmfo)[grepl("cyl", lmfo$term), ].
Component “estimate”: 'is.NA' value mismatch: 0 in current 1 in target
Component “std.error”: 'is.NA' value mismatch: 0 in current 1 in target
Component “statistic”: 'is.NA' value mismatch: 0 in current 1 in target
Component “p.value”: 'is.NA' value mismatch: 0 in current 1 in target
Component “conf.low”: 'is.NA' value mismatch: 0 in current 1 in target
Component “conf.high”: 'is.NA' value mismatch: 0 in current 1 in target
Component “df”: 'is.NA' value mismatch: 0 in current 1 in target
── Failure ('test-lm-robust-fes.R:589:5'): FEs handle collinear covariates ─────
tidy(lmo)[grepl("cyl", lmo$term), ] not equivalent to tidy(lmfo)[grepl("cyl", lmfo$term), ].
Component “estimate”: 'is.NA' value mismatch: 0 in current 1 in target
Component “std.error”: 'is.NA' value mismatch: 0 in current 1 in target
Component “statistic”: 'is.NA' value mismatch: 0 in current 1 in target
Component “p.value”: 'is.NA' value mismatch: 0 in current 1 in target
Component “conf.low”: 'is.NA' value mismatch: 0 in current 1 in target
Component “conf.high”: 'is.NA' value mismatch: 0 in current 1 in target
Component “df”: 'is.NA' value mismatch: 0 in current 1 in target
── Failure ('test-lm-robust-fes.R:589:5'): FEs handle collinear covariates ─────
tidy(lmo)[grepl("cyl", lmo$term), ] not equivalent to tidy(lmfo)[grepl("cyl", lmfo$term), ].
Component “estimate”: 'is.NA' value mismatch: 0 in current 1 in target
Component “std.error”: 'is.NA' value mismatch: 0 in current 1 in target
Component “statistic”: 'is.NA' value mismatch: 0 in current 1 in target
Component “p.value”: 'is.NA' value mismatch: 0 in current 1 in target
Component “conf.low”: 'is.NA' value mismatch: 0 in current 1 in target
Component “conf.high”: 'is.NA' value mismatch: 0 in current 1 in target
Component “df”: 'is.NA' value mismatch: 0 in current 1 in target
── Failure ('test-lm-robust-fes.R:622:3'): Handle perfect fits appropriately ───
tidy(rfo)[rfo$term == "X", ] not equivalent to tidy(ro)[ro$term == "X", ].
Component “std.error”: 'is.NA' value mismatch: 1 in current 0 in target
Component “statistic”: 'is.NA' value mismatch: 1 in current 0 in target
Component “p.value”: 'is.NA' value mismatch: 1 in current 0 in target
Component “conf.low”: 'is.NA' value mismatch: 1 in current 0 in target
Component “conf.high”: 'is.NA' value mismatch: 1 in current 0 in target

[ FAIL 4 | WARN 188 | SKIP 11 | PASS 930 ]
Error: Test failures

It would be great if you could have a look. Kind regards, Andreas.

barracuda156 commented 7 months ago

Just got the same issue on PowerPC 32-bit:


R version 4.3.3 (2024-02-29) -- "Angel Food Cake"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: powerpc-apple-darwin10.0.0d2 (32-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(testthat)
> library(estimatr)
> 
> test_check("estimatr")
[ FAIL 4 | WARN 0 | SKIP 5 | PASS 1270 ]

══ Skipped tests (5) ═══════════════════════════════════════════════════════════
• On CRAN (3): 'test-iv-robust.R:235:3', 'test-lm-cluster.R:182:3',
  'test-lm-cluster.R:213:3'
• ivpack cannot be loaded (2): 'test-iv-robust.R:30:3',
  'test-iv-robust.R:328:3'

══ Failed tests ════════════════════════════════════════════════════════════════
── Failure ('test-lm-robust-fes.R:589:5'): FEs handle collinear covariates ─────
tidy(lmo)[grepl("cyl", lmo$term), ] not equivalent to tidy(lmfo)[grepl("cyl", lmfo$term), ].
Component "estimate": 'is.NA' value mismatch: 0 in current 1 in target
Component "std.error": 'is.NA' value mismatch: 0 in current 1 in target
Component "statistic": 'is.NA' value mismatch: 0 in current 1 in target
Component "p.value": 'is.NA' value mismatch: 0 in current 1 in target
Component "conf.low": 'is.NA' value mismatch: 0 in current 1 in target
Component "conf.high": 'is.NA' value mismatch: 0 in current 1 in target
Component "df": 'is.NA' value mismatch: 0 in current 1 in target
── Failure ('test-lm-robust-fes.R:589:5'): FEs handle collinear covariates ─────
tidy(lmo)[grepl("cyl", lmo$term), ] not equivalent to tidy(lmfo)[grepl("cyl", lmfo$term), ].
Component "estimate": 'is.NA' value mismatch: 0 in current 1 in target
Component "std.error": 'is.NA' value mismatch: 0 in current 1 in target
Component "statistic": 'is.NA' value mismatch: 0 in current 1 in target
Component "p.value": 'is.NA' value mismatch: 0 in current 1 in target
Component "conf.low": 'is.NA' value mismatch: 0 in current 1 in target
Component "conf.high": 'is.NA' value mismatch: 0 in current 1 in target
Component "df": 'is.NA' value mismatch: 0 in current 1 in target
── Failure ('test-lm-robust-fes.R:589:5'): FEs handle collinear covariates ─────
tidy(lmo)[grepl("cyl", lmo$term), ] not equivalent to tidy(lmfo)[grepl("cyl", lmfo$term), ].
Component "estimate": 'is.NA' value mismatch: 0 in current 1 in target
Component "std.error": 'is.NA' value mismatch: 0 in current 1 in target
Component "statistic": 'is.NA' value mismatch: 0 in current 1 in target
Component "p.value": 'is.NA' value mismatch: 0 in current 1 in target
Component "conf.low": 'is.NA' value mismatch: 0 in current 1 in target
Component "conf.high": 'is.NA' value mismatch: 0 in current 1 in target
Component "df": 'is.NA' value mismatch: 0 in current 1 in target
── Failure ('test-lm-robust-fes.R:622:3'): Handle perfect fits appropriately ───
tidy(rfo)[rfo$term == "X", ] not equivalent to tidy(ro)[ro$term == "X", ].
Component "std.error": 'is.NA' value mismatch: 1 in current 0 in target
Component "statistic": 'is.NA' value mismatch: 1 in current 0 in target
Component "p.value": 'is.NA' value mismatch: 1 in current 0 in target
Component "conf.low": 'is.NA' value mismatch: 1 in current 0 in target
Component "conf.high": 'is.NA' value mismatch: 1 in current 0 in target

[ FAIL 4 | WARN 0 | SKIP 5 | PASS 1270 ]
Error: Test failures
Execution halted
tillea commented 7 months ago

Any idea how to fix the test suite?

barracuda156 commented 7 months ago

Any idea how to fix the test suite?

Probably expect_equal should have tolerance being set. Something like in https://github.com/kevhuy/WALS/commit/5c8c57d825eb6fabc1d88b3d898f4ccf270e1557

P. S. Perhaps the issue is rather be renamed to 32-bit instead of i386.

tillea commented 7 months ago

Probably expect_equal should have tolerance being set. Something like in kevhuy/WALS@5c8c57d

Probably, but I did not wanted to do this without confirmation by upstream.

P. S. Perhaps the issue is rather be renamed to 32-bit instead of i386.

Not really since the testsuite is passing on armel and armhf. I've changed the issue title to what we really know.