poissonconsulting / mcmcr

An R package to manipulate MCMC samples
https://poissonconsulting.github.io/mcmcr/
Other
17 stars 2 forks source link

Several tests fail on PowerPC (32-bit) due to rounding: [ FAIL 14 | WARN 11 | SKIP 3 | PASS 404 ] #56

Closed barracuda156 closed 11 months ago

barracuda156 commented 1 year ago
R version 4.2.3 (2023-03-15) -- "Shortstop Beagle"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: powerpc-apple-darwin10.8.0 (32-bit)

> library(testthat)
> library(mcmcr)
> 
> test_check("mcmcr")
[ FAIL 14 | WARN 11 | SKIP 3 | PASS 404 ]

══ Skipped tests ═══════════════════════════════════════════════════════════════
• On CRAN (3)

══ Failed tests ════════════════════════════════════════════════════════════════
── Failure ('test-rhat.R:30:3'): rhat.mcmc.list ────────────────────────────────
rhat(coda::as.mcmc.list(mcmcr_example)) (`actual`) not identical to 2.002 (`expected`).

  `actual`: 2.0019999999999998
`expected`: 2.0020000000000002
── Failure ('test-rhat.R:34:3'): rhat.mcmcmarray ───────────────────────────────
rhat(mcmcr_example[[1]], by = "term") (`actual`) not identical to c(2.002, 2.002) (`expected`).

  `actual`: 2.0019999999999998 2.0019999999999998
`expected`: 2.0020000000000002 2.0020000000000002
── Failure ('test-rhat.R:36:3'): rhat.mcmcmarray ───────────────────────────────
rhat(mcmcr_example[[1]], as_df = TRUE, by = "all") (`actual`) not identical to structure(...) (`expected`).

  `actual$rhat`: 2.0019999999999998
`expected$rhat`: 2.0020000000000002
── Failure ('test-rhat.R:54:3'): rhat.mcmcr ────────────────────────────────────
rhat(mcmcr_example2) (`actual`) not identical to 2.002 (`expected`).

  `actual`: 2.0019999999999998
`expected`: 2.0020000000000002
── Failure ('test-rhat.R:55:3'): rhat.mcmcr ────────────────────────────────────
rhat(mcmcr_example, as_df = TRUE) (`actual`) not identical to structure(...) (`expected`).

  `actual$rhat`: 2.0019999999999998
`expected$rhat`: 2.0020000000000002
── Failure ('test-rhat.R:63:3'): rhat.mcmcr ────────────────────────────────────
rhat(mcmcr_example) (`actual`) not identical to 2.002 (`expected`).

  `actual`: 2.0019999999999998
`expected`: 2.0020000000000002
── Failure ('test-rhat.R:64:3'): rhat.mcmcr ────────────────────────────────────
rhat(mcmcr_example, by = "parameter") (`actual`) not identical to list(alpha = 2.002, beta = 1.147, sigma = 1) (`expected`).

  `actual$alpha`: 2.0019999999999998
`expected$alpha`: 2.0020000000000002
── Failure ('test-rhat.R:80:3'): rhat.mcmcrs ───────────────────────────────────
rhat(mcmcrs(mcmcr_example, mcmcr_example)) (`actual`) not identical to list(mcmcr1 = 2.002, mcmcr2 = 2.002) (`expected`).

  `actual$mcmcr1`: 2.0019999999999998
`expected$mcmcr1`: 2.0020000000000002

  `actual$mcmcr2`: 2.0019999999999998
`expected$mcmcr2`: 2.0020000000000002
── Failure ('test-rhat.R:81:3'): rhat.mcmcrs ───────────────────────────────────
rhat(mcmcrs(mcmcr_example, mcmcr_example), bound = TRUE) (`actual`) not identical to list(mcmcr1 = 2.002, mcmcr2 = 2.002, bound = 1.891) (`expected`).

  `actual$mcmcr1`: 2.0019999999999998
`expected$mcmcr1`: 2.0020000000000002

  `actual$mcmcr2`: 2.0019999999999998
`expected$mcmcr2`: 2.0020000000000002
── Failure ('test-rhat.R:82:3'): rhat.mcmcrs ───────────────────────────────────
rhat(mcmcrs(mcmcr_example, mcmcr_example), by = "parameter") (`actual`) not identical to list(...) (`expected`).

  `actual$mcmcr1$alpha`: 2.0019999999999998
`expected$mcmcr1$alpha`: 2.0020000000000002

  `actual$mcmcr2$alpha`: 2.0019999999999998
`expected$mcmcr2$alpha`: 2.0020000000000002
── Failure ('test-rhat.R:89:3'): rhat.mcmcrs ───────────────────────────────────
rhat(mcmcrs(mcmcr_example, mcmcr_example), bound = TRUE, by = "parameter") (`actual`) not identical to list(...) (`expected`).

  `actual$mcmcr1$alpha`: 2.0019999999999998
`expected$mcmcr1$alpha`: 2.0020000000000002

  `actual$mcmcr2$alpha`: 2.0019999999999998
`expected$mcmcr2$alpha`: 2.0020000000000002
── Failure ('test-rhat.R:95:3'): rhat.mcmcrs ───────────────────────────────────
rhat(mcmcrs(mcmcr_example, mcmcr_example), bound = TRUE, as_df = TRUE) (`actual`) not identical to structure(...) (`expected`).

  `actual$mcmcr1`: 2.0019999999999998
`expected$mcmcr1`: 2.0020000000000002

  `actual$mcmcr2`: 2.0019999999999998
`expected$mcmcr2`: 2.0020000000000002
── Failure ('test-rhat.R:96:3'): rhat.mcmcrs ───────────────────────────────────
rhat(...) (`actual`) not identical to structure(...) (`expected`).

  `actual$mcmcr1`: 2.0019999999999998 1.1470000000000000 1.0000000000000000
`expected$mcmcr1`: 2.0020000000000002 1.1470000000000000 1.0000000000000000

  `actual$mcmcr2`: 2.0019999999999998 1.1470000000000000 1.0000000000000000
`expected$mcmcr2`: 2.0020000000000002 1.1470000000000000 1.0000000000000000
── Failure ('test-rhat.R:97:3'): rhat.mcmcrs ───────────────────────────────────
rhat(...) (`actual`) not identical to structure(...) (`expected`).

    actual$mcmcr1      | expected$mcmcr1       
[1] 2.0019999999999998 - 2.0020000000000002 [1]
[2] 2.0019999999999998 - 2.0020000000000002 [2]
[3] 1.1470000000000000 | 1.1470000000000000 [3]
[4] 1.1470000000000000 | 1.1470000000000000 [4]
[5] 1.1470000000000000 | 1.1470000000000000 [5]

    actual$mcmcr2      | expected$mcmcr2       
[1] 2.0019999999999998 - 2.0020000000000002 [1]
[2] 2.0019999999999998 - 2.0020000000000002 [2]
[3] 1.1470000000000000 | 1.1470000000000000 [3]
[4] 1.1470000000000000 | 1.1470000000000000 [4]
[5] 1.1470000000000000 | 1.1470000000000000 [5]

[ FAIL 14 | WARN 11 | SKIP 3 | PASS 404 ]
Deleting unused snapshots:
• converged/converged.csv
• ess/ess.csv
• estimates/estimates.csv
Error: Test failures
Execution halted
joethorley commented 1 year ago

@barracuda156 - thanks for the report!

barracuda156 commented 1 year ago

@barracuda156 - thanks for the report!

@joethorley Is this something that we could fix?

P. S. I have encountered rounding issues with a couple of other R packages, but no solution so far. It seems that at least in some cases it is neither PPC nor 32-bit issue, but rather something caused by MacOS handling.

joethorley commented 1 year ago

@barracuda156 If you are able to identify the exact cause of the rounding issues that would be extremely helpful. Otherwise I just have to reduce the number of significant digits on the tests which is extremely unsatisfactory.

barracuda156 commented 1 year ago

@barracuda156 If you are able to identify the exact cause of the rounding issues that would be extremely helpful. Otherwise I just have to reduce the number of significant digits on the tests which is extremely unsatisfactory.

I have ran tests on 10.6.8/x86_64, everything passes, so in this case it is apparently 32-bit issue.

joethorley commented 1 year ago

That is good to know!

joethorley commented 11 months ago

@barracuda156 - I've updated the tests so that by default they only test data snapshots to 6 significant figures.

Can you test if mcmcr now passes on PowerPC 32 bit?

barracuda156 commented 11 months ago

@joethorley Sure, will try tonight. Thank you!

barracuda156 commented 11 months ago

@joethorley Yes, all good now, thank you for addressing this!

R version 4.3.2 (2023-10-31) -- "Eye Holes"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: powerpc-apple-darwin10.0.0d2 (32-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(testthat)
> library(mcmcr)
> 
> test_check("mcmcr")
[ FAIL 0 | WARN 0 | SKIP 3 | PASS 436 ]

══ Skipped tests (3) ═══════════════════════════════════════════════════════════
• On CRAN (3): 'test-converged.R:31:3', 'test-ess.R:23:3',
  'test-estimates.R:4:3'

[ FAIL 0 | WARN 0 | SKIP 3 | PASS 436 ]
Deleting unused snapshots:
• converged/converged.csv
• ess/ess.csv
• estimates/estimates.csv
> 
> proc.time()
   user  system elapsed 
 47.322   1.892  55.528 
joethorley commented 11 months ago

That's good news!