search

tidyverse / tibble

A modern re-imagining of the data frame
https://tibble.tidyverse.org/
Other
671 stars 130 forks source link

tibble 3.2.1 testing fails on Windows R-4.3.2 #1576

Open olathan opened 5 months ago

olathan commented 5 months ago

I have problems to run the testing of tibble 3.2.1 on R-4.3.2 in a Windows environment (knitr is installed in version 1.45).

The following test in the script "test-vignette-formats.R" fails:

test_that("formats vignette", {
  skip_if_not_installed("knitr", "1.34.2")

  # Fails on Linux
  test_galley("formats")
})
== Failed tests ================================================================
-- Error ('test-vignette-formats.R:5:3'): formats vignette ---------------------
<callr_status_error/callr_error/rlib_error_3_0/rlib_error/error/condition>
Error: ! in callr subprocess.
Caused by error in `value[[3L]](cond)`:
! Could not find a root 'DESCRIPTION' file that starts with '^Package' in '...\tibble-tests\testthat'.

After copying the tibble 'DESCRIPTION' from

"...\R-4.3.2\library\tibble\DESCRIPTION"

to the testthat directory

"...\R-4.3.2\library\tibble\tests\testthat\DESCRIPTION"

I got this fail:

== Failed tests ================================================================
-- Error ('test-vignette-formats.R:5:3'): formats vignette ---------------------
<callr_status_error/callr_error/rlib_error_3_0/rlib_error/error/condition>
Error: ! in callr subprocess.
Caused by error in `abs_path(input)`:
! The file '' does not exist.
Backtrace:
    x
 1. \-tibble:::test_galley("formats") at test-vignette-formats.R:5:3
 2.   \-tibble:::render_galley(rmd_name, md_name)
 3.     \-base::tryCatch(...)
 4.       \-base (local) tryCatchList(expr, classes, parentenv, handlers)
 5.         \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
 6.           \-value[[3L]](cond)
krlmlr commented 5 months ago

Thanks. Can you please describe your workflow for testing? R CMD check works on GitHub Actions on Windows: https://github.com/tidyverse/tibble/actions/runs/9040049629/job/24843799253 .

olathan commented 5 months ago

Thanks for testing on GitHub.

R version 4.3.2 (2023-10-31 ucrt) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows Server 2019 x64 (build 17763) Change access to all folders

Tibble version 3.2.1 https://cran.r-project.org/src/contrib/tibble_3.2.1.tar.gz Installed with (I shortened the folder name):

install.packages("...\tibble_3.2.1.tar.gz", repos = NULL, type ="source", lib = "...\R-4.3.2\", INSTALL_opts = "--no-test-load")
DONE (tibble)

```r * installing *source* package 'tibble' ... ** package 'tibble' successfully unpacked and MD5 sums checked ** using staged installation ** libs using C compiler: 'gcc.exe (GCC) 12.2.0' gcc -I"q:/Validation/R_package/R-4.3.2/include" -DNDEBUG -std=gnu99 -I"p:/StatsTools/R_package/rtools43/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c attributes.c -o attributes.o gcc -I"q:/Validation/R_package/R-4.3.2/include" -DNDEBUG -std=gnu99 -I"p:/StatsTools/R_package/rtools43/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c coerce.c -o coerce.o gcc -I"q:/Validation/R_package/R-4.3.2/include" -DNDEBUG -std=gnu99 -I"p:/StatsTools/R_package/rtools43/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c init.c -o init.o gcc -I"q:/Validation/R_package/R-4.3.2/include" -DNDEBUG -std=gnu99 -I"p:/StatsTools/R_package/rtools43/x86_64-w64-mingw32.static.posix/include" -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c matrixToDataFrame.c -o matrixToDataFrame.o gcc -shared -s -static-libgcc -o tibble.dll tmp.def attributes.o coerce.o init.o matrixToDataFrame.o -Lp:/StatsTools/R_package/rtools43/x86_64-w64-mingw32.static.posix/lib/x64 -Lp:/StatsTools/R_package/rtools43/x86_64-w64-mingw32.static.posix/lib -Lq:/Validation/R_package/R-4.3.2/bin/x64 -lR installing to q:/Validation/R_package/R-4.3.2/library/00LOCK-tibble/00new/tibble/libs/x64 ** R ** inst ** byte-compile and prepare package for lazy loading ** help *** installing help indices *** copying figures ** building package indices ** installing vignettes * DONE (tibble) ```

Note: In my environment I need to run "source" package installation with option: INSTALL_opts = "--no-test-load" Otherwise I would get Error: loading failed and Warning in library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE) : there is no package called 'tibble'

Than I copy the tests folder to the library with xcopy "...\tibble_3.2.1.tar.gz\tests*" "...\R-4.3.2\library\tibble\tests\" /s /i /y This installation and test copy process works fine for more than 30 other packages, so I would not expect problems here. Additionally I had to copy "...\R-4.3.2\library\tibble\DESCRIPTION" to the testthat directory "...\R-4.3.2\library\tibble\tests\testthat\DESCRIPTION" (as described above)

Running the test in a windows command:

"...\R.exe" CMD BATCH --vanilla "...\TestScript.R" "...\TestScript.Rout"

TestScript.R:

pdf("tests.pdf")
Sys.setenv(LC_COLLATE = "C", LC_TIME = "C", LANGUAGE = "en")
Sys.setenv(NOT_CRAN = "true")
setwd("q:/Validation/R_package/TestingDocumentation/R/TestCaseScriptOutput/TC5")
Sys.getenv()

tools::testInstalledPackage(pkg = 'tibble'),
  lib.loc = "q:/Validation/R_package/R-4.3.2/library",
  outDir = "q:/Validation/R_package/TestingDocumentation/R/TestCaseScriptOutput/TC5",
  types = "tests",
  srcdir = "q:/Validation/R_package/TestingDocumentation/R/TestCaseScriptOutput/TC5")
Sys.getenv() gives the following output:

```r AdditionalTests q:\Validation\R_package\TestingDocumentation\R\TestCaseScript\AdditionalTests ALLUSERSPROFILE C:\ProgramData APPDATA \\company.de\metro\Home\me\CitrixFDRProfile\AppData BIBINPUTS .;;q:/Validation/R_package/R-4.3.2/share/texmf/bibtex/bib; BSTINPUTS .;;q:/Validation/R_package/R-4.3.2/share/texmf/bibtex/bst; CLIENTNAME METROLAPTOP023 COMMANDER_DRIVE C: COMMANDER_EXE C:\totalcmd\TOTALCMD64.EXE COMMANDER_INI \\company.de\metro\Home\me\CitrixFDRProfile\AppData\GHISLER\wincmd.ini COMMANDER_INI_PATH \\company.de\metro\Home\me\CitrixFDRProfile\AppData\GHISLER COMMANDER_INSTANCE 1 COMMANDER_OPEN open COMMANDER_PATH C:\totalcmd COMMANDER_SESSIONNAME user CommonProgramFiles C:\Program Files\Common Files CommonProgramFiles(x86) C:\Program Files (x86)\Common Files CommonProgramW6432 C:\Program Files\Common Files COMPUTERNAME METROO36503 ComSpec C:\WINDOWS\system32\cmd.exe CURL_CA_BUNDLE q:/Validation/R_package/R-4.3.2/etc/curl-ca-bundle.crt DriverData C:\Windows\System32\Drivers\DriverData FPS_BROWSER_APP_PROFILE_STRING Internet Explorer FPS_BROWSER_USER_PROFILE_STRING Default GlobalFolder Q:\Validation\R_package\TestingDocumentation\R\ GlobalFolderR Q:/Validation/R_package/TestingDocumentation/R/ HOME \\company.de\metro\Home\me\Documents HOMEDRIVE C: HOMEPATH \Users\me LANGUAGE en LC_COLLATE C LC_TIME C LOCALAPPDATA C:\Users\me\AppData\Local LOGONSERVER \\METRODC1 MSYS2_ENV_CONV_EXCL R_ARCH NOT_CRAN true NUMBER_OF_PROCESSORS 6 ORACLE_HOME C:\Oracle\OracleClient11 OS Windows_NT PATH p:\StatsTools\R_package\rtools43/x86_64-w64-mingw32.static.posix/bin;p:\StatsTools\R_package\rtools43/usr/bin;p:\StatsTools\R_package\rtools43/x86_64-w64-mingw32.static.posix/bin;p:\StatsTools\R_package\rtools43/usr/bin;C:\Program Files\AdoptOpenJDK\jdk-8.0.292.10-hotspot\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\Oracle\OracleClient11\bin;C:\Program Files\dotnet\;C:\Program Files\Git\cmd;C:\Program Files\Microsoft VS Code\bin;C:\Program Files\MiKTeX\miktex\bin\x64\;C:\Program Files (x86)\Citrix\HDX\bin\;C:\Program Files\Citrix\HDX\bin\;C:\Program Files\IDM Computer Solutions\UltraCompare\;C:\Program Files\TortoiseSVN\bin;C:\Users\me\AppData\Local\Microsoft\WindowsApps;/ PATHEXT .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE AMD64 PROCESSOR_IDENTIFIER Intel64 Family 6 Model 85 Stepping 0, GenuineIntel PROCESSOR_LEVEL 6 PROCESSOR_REVISION 5500 ProgramData C:\ProgramData ProgramFiles C:\Program Files ProgramFiles(x86) C:\Program Files (x86) ProgramW6432 C:\Program Files PROMPT $P$G PSModulePath C:\Program Files\WindowsPowerShell\Modules;C:\WINDOWS\system32\WindowsPowerShell\v1.0\Modules;C:\Program Files\Citrix\Telemetry Service\ PUBLIC C:\Users\Public R_ARCH /x64 R_BZIPCMD bzip2 R_COMPILED_BY gcc 12.3.0 R_DOC_DIR q:/Validation/R_package/R-4.3.2/doc R_GZIPCMD gzip R_HOME q:/Validation/R_package/R-4.3.2 R_INCLUDE_DIR q:/Validation/R_package/R-4.3.2/include R_LIBS_SITE q:\Validation\R_package\R-4.3.2_SiteLib R_LIBS_USER C:\Users\me\AppData\Local/R/win-library/4.3 R_OSTYPE windows R_PAPERSIZE a4 R_RD4PDF times,inconsolata,hyper R_RTOOLS43_PATH p:\StatsTools\R_package\rtools43/x86_64-w64-mingw32.static.posix/bin;p:\StatsTools\R_package\rtools43/usr/bin R_SHARE_DIR q:/Validation/R_package/R-4.3.2/share R_UNZIPCMD unzip R_USER \\company.de\metro\Home\me\Documents R_ZIPCMD zip RPrg q:\Validation\R_package\R-4.3.2 RTOOLS43_HOME p:\StatsTools\R_package\rtools43 script TC5 SED sed SESSIONNAME ICA-CGP#1 SystemDrive C: SystemRoot C:\WINDOWS TC1 rem TC2 rem TC3 rem TC4 rem TC6 rem TCX rem TEMP C:\Users\LOCAL_~2\Temp\3 TEXINPUTS .;;q:/Validation/R_package/R-4.3.2/share/texmf/tex/latex; TMP C:\Users\LOCAL_~2\Temp\3 TNS_ADMIN C:\Oracle\OracleClient11\Network\Admin USERDNSDOMAIN COMPANY.DE USERDOMAIN HEADQUARTERS USERDOMAIN_ROAMINGPROFILE HEADQUARTERS USERNAME me USERPROFILE C:\Users\me windir C:\WINDOWS > > tools::testInstalledPackage(pkg = 'tibble', + lib.loc = "q:/Validation/R_package/R-4.3.2/library", + outDir = "q:/Validation/R_package/TestingDocumentation/R/TestCaseScriptOutput/TC5", + types = "tests", + srcdir = "q:/Validation/R_package/TestingDocumentation/R/TestCaseScriptOutput/TC5") Running specific tests for package 'tibble' Running 'testthat.R' > ```

testthat.Rout.Fail

```r R version 4.3.2 (2023-10-31 ucrt) -- "Eye Holes" Copyright (C) 2023 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit) R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details. Natural language support but running in an English locale R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications. Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R. > library("testthat") > > test_check("tibble") Loading required package: tibble Starting 2 test processes [ FAIL 10 | WARN 0 | SKIP 1 | PASS 969 ] == Skipped tests (1) =========================================================== * On Windows (1): 'test-print.R:56:3' == Failed tests ================================================================ -- Error ('test-vignette-formats.R:5:3'): formats vignette --------------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("formats") at test-vignette-formats.R:5:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) -- Failure ('test-print.R:11:3'): output test ---------------------------------- Snapshot of code has changed: old | new [4] Output | Output [4] [5] # A tibble: 32 x 11 | # A tibble: 32 x 11 [5] [6] | [6] [7] # ... with 24 more rows, and - # i 24 more rows [7] [8] # 7 more variables: - # i 7 more variables: [8] [9] # drat , wt , | # drat , wt , [9] [10] # qsec , vs , | # qsec , vs , [10] [11] # am , gear , | # am , gear , [11] old[15:27] vs new[15:27] Output # A tibble: 31 x 3 - # ... with 26 more rows + # i 26 more rows Code print_without_body(as_tibble(trees), n = -1L, width = 30L) Output # A tibble: 31 x 3 - # ... with 21 more rows + # i 21 more rows and 3 more ... old | new [28] # A tibble: 31 | # A tibble: 31 [28] [29] # x 3 | # x 3 [29] [30] | [30] [31] # ... with 1 - # i 1 more [31] [32] # more - [33] # variable: | # variable: [32] [34] # Volume | # Volume [33] [35] Code | Code [34] old[37:49] vs new[36:48] Output # A tibble: 31 x 3 - # ... with 21 more rows + # i 21 more rows Code print_without_body(as_unknown_rows(trees), n = 10, width = 70L) Output # A tibble: ?? x 3 - # ... with more rows + # i more rows and 3 more ... old[59:71] vs new[58:70] Output # A tibble: ?? x 3 - # ... with more rows + # i more rows Code print_without_body(df_all, n = NULL, width = 30L) Output # A tibble: 3 x 9 - # ... with 5 more variables: + # i 5 more variables: and 3 more ... old[79:92] vs new[78:91] Output # A tibble: 10,000 x 1 - # ... with 9,995 more rows + # i 9,995 more rows Code print_without_body(tibble(a = character(), b = logical()), width = 30L) Output # A tibble: 0 x 2 - # ... with 2 variables: + # i 2 variables: a , and 4 more ... old | new [97] Output | Output [96] [98] # A tibble: ?? x 3 | # A tibble: ?? x 3 [97] [99] | [98] [100] # ... with 3 variables: - # i 3 variables: Girth , [99] [101] # Girth , - [102] # Height , | # Height , [100] [103] # Volume | # Volume [101] [104] Code | Code [102] old[106:116] vs new[104:113] Output # A tibble: ?? x 0 - # ... with at least 5 rows - # total + # i at least 5 rows total Code print_without_body(as_unknown_rows(tibble(a = seq.int(10000))), n = 5L, width = 30L) Output # A tibble: ?? x 1 and 1 more ... * Run `testthat::snapshot_accept('print')` to accept the change. * Run `testthat::snapshot_review('print')` to interactively review the change. -- Failure ('test-tbl_df.R:77:3'): output test --------------------------------- Snapshot of code has changed: old[3:19] vs new[3:19] names(df) <- NULL Condition Warning: - The `value` argument of `names<-` can't be NULL as of tibble 3.0.0. + The `value` argument of `names<-()` can't be NULL as of tibble 3.0.0. Warning: - The `value` argument of `names<-` must have the same length as `x` as of tibble 3.0.0. + The `value` argument of `names<-()` must have the same length as `x` as of tibble 3.0.0. Warning: - The `value` argument of `names<-` can't be empty as of tibble 3.0.0. + The `value` argument of `names<-()` can't be empty as of tibble 3.0.0. Code names(df) <- "c" and 7 more ... * Run `testthat::snapshot_accept('tbl_df')` to accept the change. * Run `testthat::snapshot_review('tbl_df')` to interactively review the change. -- Error ('test-vignette-digits.R:2:3'): digits vignette ----------------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("digits") at test-vignette-digits.R:2:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) -- Error ('test-vignette-extending.R:2:3'): extending vignette ----------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("extending") at test-vignette-extending.R:2:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) -- Error ('test-vignette-invariants.R:5:3'): invariants vignette --------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("invariants", variant = rlang_pillar_variant()) at test-vignette-invariants.R:5:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) -- Error ('test-vignette-numbers.R:2:3'): numbers vignette --------------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("numbers") at test-vignette-numbers.R:2:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) -- Error ('test-vignette-tibble.R:5:3'): tibble vignette ----------------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("tibble", variant = rlang_variant()) at test-vignette-tibble.R:5:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) -- Failure ('test-subsetting.R:762:3'): output test ---------------------------- Snapshot of code has changed: old[108:114] vs new[108:114] invisible(foo[1:5, ]) Condition Warning: - The `i` argument of `[.tbl_df` must lie in [0, rows] if positive, as of tibble 3.0.0. + The `i` argument of `[.tbl_df()` must lie in [0, rows] if positive, as of tibble 3.0.0. i Use `NA_integer_` as row index to obtain a row full of `NA` values. Code foo[-1:1, ] old[135:141] vs new[135:141] invisible(foo[-4, ]) Condition Warning: - The `i` argument of `[.tbl_df` must lie in [-rows, 0] if negative, as of tibble 3.0.0. + The `i` argument of `[.tbl_df()` must lie in [-rows, 0] if negative, as of tibble 3.0.0. i Use `NA_integer_` as row index to obtain a row full of `NA` values. Code foo[array(1, dim = c(1, 1, 1)), ] old[286:304] vs new[286:304] invisible(foo[3:5, ]) Condition Warning: - The `i` argument of `[.tbl_df` must lie in [0, rows] if positive, as of tibble 3.0.0. + The `i` argument of `[.tbl_df()` must lie in [0, rows] if positive, as of tibble 3.0.0. i Use `NA_integer_` as row index to obtain a row full of `NA` values. Code invisible(foo[-(3:5), ]) Condition Warning: - The `i` argument of `[.tbl_df` must lie in [-rows, 0] if negative, as of tibble 3.0.0. + The `i` argument of `[.tbl_df()` must lie in [-rows, 0] if negative, as of tibble 3.0.0. and 9 more ... old[436:448] vs new[436:448] invisible(foo[matrix(1:2, ncol = 1), ]) Condition Warning: - The `i` argument of `[` can't be a matrix as of tibble 3.0.0. + The `i` argument of `[()` can't be a matrix as of tibble 3.0.0. i Convert to a vector. Code invisible(foo[matrix(rep(TRUE, 10), ncol = 1), ]) Condition Warning: - The `i` argument of `[` can't be a matrix as of tibble 3.0.0. + The `i` argument of `[()` can't be a matrix as of tibble 3.0.0. and 3 more ... * Run `testthat::snapshot_accept('subsetting')` to accept the change. * Run `testthat::snapshot_review('subsetting')` to interactively review the change. -- Error ('test-vignette-types.R:2:3'): types vignette ------------------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("types", variant = rlang_variant()) at test-vignette-types.R:2:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) [ FAIL 10 | WARN 0 | SKIP 1 | PASS 969 ] Deleting unused snapshots: * vignette-digits/digits.md * vignette-extending/extending.md * vignette-formats/formats.md * vignette-invariants/invariants.md * vignette-numbers/numbers.md * vignette-types/types.md Error: Test failures In addition: Warning message: package 'tibble' was built under R version 4.3.3 Execution halted ```

I could not find any further relevant information. Hope this is fine for you.

krlmlr commented 4 months ago

Thanks, this is useful.

This is a nonstandard testing procedure that violates an assumption that tibble makes on the location of files:

https://github.com/tidyverse/tibble/blob/0686ca53964163a97d5d422fac673f2574a326f1/R/galley.R#L29-L43

The error might go away if you satisfy the requirements of galley_use_installed() .

Would you be able to check a GitHub version? Happy to try out a proper fix because the error came up elsewhere too.

olathan commented 4 months ago

Yes we can try to let me check a fix...

install_github("tidyverse/tibble")
# works, but I have to copy tests manually into this lib. No problem.

The assumption on the location of files result in the same Problem with my settings for the following 7 test scripts:

'test-vignette-formats.R:3:3', 'test-vignette-digits.R:2:3', 'test-vignette-extending.R:2:3', 'test-vignette-invariants.R:2:3',
  'test-vignette-numbers.R:2:3', 'test-vignette-tibble.R:2:3', 'test-vignette-types.R:2:3'

If I had to skip all these tests for my system, it would be important that I make a risk assessment for this (Qualification of R in our environment). My guess is that all these tests are used to check the correctness of the vignettes. This would then have no risk to the correctness of tibble's results. Can you confirm this for me?

Would you mind also looking for a fix of 'DESCRIPTION' location in the same tests?

-- Error ('test-vignette-formats.R:5:3'): formats vignette ---------------------
<callr_status_error/callr_error/rlib_error_3_0/rlib_error/error/condition>
Error: ! in callr subprocess.
Caused by error in `value[[3L]](cond)`:
! Could not find a root 'DESCRIPTION' file that starts with '^Package' in '...\tibble-tests\testthat'.
krlmlr commented 4 months ago

Thanks. Can you please check if you get better results with https://github.com/tidyverse/tibble/pull/1582 ?

olathan commented 4 months ago

If I had to skip all these tests for my system, it would be important that I make a risk assessment for this (Qualification of R in our environment). My guess is that all these tests are used to check the correctness of the vignettes. This would then have no risk to the correctness of tibble's results. Can you confirm this for me?

I installed your pull request with:

library(devtools)
install_github("tidyverse/tibble", ref=github_pull(1582))

Then I added the “tests” folder and had the same problem, first with the missing DESCRIPTION file and after adding it with these errors:

Details

```r == Failed tests ================================================================ -- Error ('test-vignette-formats.R:5:3'): formats vignette --------------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("formats") at test-vignette-formats.R:5:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) -- Error ('test-vignette-digits.R:2:3'): digits vignette ----------------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("digits") at test-vignette-digits.R:2:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) -- Error ('test-vignette-extending.R:2:3'): extending vignette ----------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("extending") at test-vignette-extending.R:2:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) -- Error ('test-vignette-invariants.R:5:3'): invariants vignette --------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("invariants", variant = rlang_pillar_variant()) at test-vignette-invariants.R:5:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) -- Error ('test-vignette-numbers.R:2:3'): numbers vignette --------------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("numbers") at test-vignette-numbers.R:2:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) -- Error ('test-vignette-tibble.R:5:3'): tibble vignette ----------------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("tibble", variant = rlang_variant()) at test-vignette-tibble.R:5:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) -- Error ('test-vignette-types.R:2:3'): types vignette ------------------------- Error: ! in callr subprocess. Caused by error in `abs_path(input)`: ! The file '' does not exist. Backtrace: x 1. \-tibble:::test_galley("types", variant = rlang_variant()) at test-vignette-types.R:2:3 2. \-tibble:::render_galley(rmd_name, md_name) 3. \-base::tryCatch(...) 4. \-base (local) tryCatchList(expr, classes, parentenv, handlers) 5. \-base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6. \-value[[3L]](cond) [ FAIL 7 | WARN 0 | SKIP 1 | PASS 972 ] ```

krlmlr commented 3 months ago

Thanks. I think the bottom line is that the tests must be self-contained and not use vignettes. PR forthcoming.

krlmlr commented 3 months ago

Can you please try #1586?

olathan commented 3 months ago

I installed #1586, added the test folder from version 3.2.1 the 'DESCRIPTION' file and the 'vignettes' folder to the 'testthat' folder and tested this. Sorry I did not find the test folder of #1586.

I got these failures:

== Failed tests ================================================================ -- Error ('test-vignette-formats.R:5:3'): formats vignette --------------------- Error: ! in callr subprocess. Caused by error: ! package or namespace load failed for 'dplyr': object 'add_row' is not exported by 'namespace:tibble' -- Error ('test-vignette-digits.R:2:3'): digits vignette ----------------------- Error: ! in callr subprocess. Caused by error in `tibble(x = c(1.2345, 12.345, 123.45, 1234.5, 12345))`: ! could not find function "tibble" -- Error ('test-vignette-extending.R:2:3'): extending vignette ----------------- Error: ! in callr subprocess. Caused by error in `new_tibble(list(a = 1:3, b = 2:4), class = "my_tbl_df")`: ! could not find function "new_tibble" -- Error ('test-vignette-invariants.R:5:3'): invariants vignette --------------- Error: ! in callr subprocess. Caused by error in `eval(expr, envir, enclos)`: ! object 'set_dftbl_hooks' not found -- Error ('test-vignette-numbers.R:2:3'): numbers vignette --------------------- Error: ! in callr subprocess. Caused by error in `tibble(x = 123.4567)`: ! could not find function "tibble" -- Error ('test-vignette-tibble.R:5:3'): tibble vignette ----------------------- Error: ! in callr subprocess. Caused by error in `tibble(x = letters)`: ! could not find function "tibble" -- Error ('test-vignette-types.R:2:3'): types vignette ------------------------- Error: ! in callr subprocess. Caused by error: ! package or namespace load failed for 'dplyr': object 'add_row' is not exported by 'namespace:tibble'

krlmlr commented 3 months ago

Thanks. I double-checked, with the PR you should no longer need to copy the DESCRIPTION file. This might be harmful and the reason for the errors you're seeing.

Can you please start from scratch with this PR, using the same procedure you do for other packages, and try again?

olathan commented 3 months ago

Sorry I can't find a way to test this version with your changed tests. I just get the tibble installation without tests and copied the tests from Version 3.2.1 and this creates the error messages I described in my first comment of May 6.

krlmlr commented 3 months ago

Can you clone https://github.com/tidyverse/tibble/pull/1586, then do R CMD build . in the clone? This gives you a tarball (tibble_3.x.x.xxxx.tar.gz), pretty much like those you can also download from CRAN.