configuration failure should be reported

[tdhock]

https://rcdata.nau.edu/genomic-ml/data.table-revdeps/analyze/2023-04-11/FIESTA.txt says

FIESTAutils installs OK for R-devel

Installing package into '/tmp/th798/64531145/R-devel/392/library'
(as 'lib' is unspecified)
also installing the dependencies 'mitools', 'rbibutils', 'survey', 'rpms', 'Rdpack', 'hbsae', 'JoSAE', 'mase', 'RPostgreSQL', 'sae', 'plogr', 'gsubfn', 'proto', 'chron', 'FIESTAutils', 'RSQLite', 'sqldf', 'largeList'
...
* installing *source* package 'FIESTAutils' ...
** package 'FIESTAutils' successfully unpacked and MD5 sums checked
** using staged installation
** libs
using C++ compiler: 'g++ (GCC) 8.5.0 20210514 (Red Hat 8.5.0-15)'
g++ -std=gnu++17 -I"/home/th798/R/R-devel/include" -DNDEBUG -DRCPP_USE_UNWIND_PROTECT -I'/projects/genomic-ml/R/R-devel/library/Rcpp/include' -I/home/th798/.conda/envs/emacs1/include -I/home/th798/include    -fpic  -g -O2  -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++17 -I"/home/th798/R/R-devel/include" -DNDEBUG -DRCPP_USE_UNWIND_PROTECT -I'/projects/genomic-ml/R/R-devel/library/Rcpp/include' -I/home/th798/.conda/envs/emacs1/include -I/home/th798/include    -fpic  -g -O2  -c r_cmb_table.cpp -o r_cmb_table.o
g++ -std=gnu++17 -I"/home/th798/R/R-devel/include" -DNDEBUG -DRCPP_USE_UNWIND_PROTECT -I'/projects/genomic-ml/R/R-devel/library/Rcpp/include' -I/home/th798/.conda/envs/emacs1/include -I/home/th798/include    -fpic  -g -O2  -c r_rasterize.cpp -o r_rasterize.o
g++ -std=gnu++17 -I"/home/th798/R/R-devel/include" -DNDEBUG -DRCPP_USE_UNWIND_PROTECT -I'/projects/genomic-ml/R/R-devel/library/Rcpp/include' -I/home/th798/.conda/envs/emacs1/include -I/home/th798/include    -fpic  -g -O2  -c r_running_stats.cpp -o r_running_stats.o
g++ -std=gnu++17 -shared -L/home/th798/.conda/envs/emacs1/lib -Wl,-rpath=/home/th798/.conda/envs/emacs1/lib -L/home/th798/lib -Wl,-rpath=/home/th798/lib -L/home/th798/lib64 -Wl,-rpath=/home/th798/lib64 -o FIESTAutils.so RcppExports.o r_cmb_table.o r_rasterize.o r_running_stats.o
installing to /tmp/th798/64531145/R-devel/392/library/00LOCK-FIESTAutils/00new/FIESTAutils/libs
** R
** data
*** moving datasets to lazyload DB
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (FIESTAutils)

but it fails for R-release:

Installing package into '/tmp/th798/64531145/R-4.2.3/392/library'
(as 'lib' is unspecified)
also installing the dependencies 'rbibutils', 'survey', 'rpms', 'Rdpack', 'hbsae', 'JoSAE', 'mase', 'RPostgreSQL', 'sae', 'plogr', 'gsubfn', 'proto', 'chron', 'FIESTAutils', 'RSQLite', 'sqldf', 'largeList'
...
* installing *source* package 'FIESTAutils' ...
** package 'FIESTAutils' successfully unpacked and MD5 sums checked
** using staged installation
** libs
g++ -std=gnu++14 -I"/home/th798/R/R-4.2.3/include" -DNDEBUG -DRCPP_USE_UNWIND_PROTECT -I'/projects/genomic-ml/R/R-4.2.3/library/Rcpp/include' -I/home/th798/.conda/envs/emacs1/include -I/home/th798/include   -fpic  -g -O2  -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++14 -I"/home/th798/R/R-4.2.3/include" -DNDEBUG -DRCPP_USE_UNWIND_PROTECT -I'/projects/genomic-ml/R/R-4.2.3/library/Rcpp/include' -I/home/th798/.conda/envs/emacs1/include -I/home/th798/include   -fpic  -g -O2  -c r_cmb_table.cpp -o r_cmb_table.o
g++ -std=gnu++14 -I"/home/th798/R/R-4.2.3/include" -DNDEBUG -DRCPP_USE_UNWIND_PROTECT -I'/projects/genomic-ml/R/R-4.2.3/library/Rcpp/include' -I/home/th798/.conda/envs/emacs1/include -I/home/th798/include   -fpic  -g -O2  -c r_rasterize.cpp -o r_rasterize.o
g++ -std=gnu++14 -I"/home/th798/R/R-4.2.3/include" -DNDEBUG -DRCPP_USE_UNWIND_PROTECT -I'/projects/genomic-ml/R/R-4.2.3/library/Rcpp/include' -I/home/th798/.conda/envs/emacs1/include -I/home/th798/include   -fpic  -g -O2  -c r_running_stats.cpp -o r_running_stats.o
g++ -std=gnu++14 -shared -L/home/th798/.conda/envs/emacs1/lib -Wl,-rpath=/home/th798/.conda/envs/emacs1/lib -L/home/th798/lib -Wl,-rpath=/home/th798/lib -L/home/th798/lib64 -Wl,-rpath=/home/th798/lib64 -o FIESTAutils.so RcppExports.o r_cmb_table.o r_rasterize.o r_running_stats.o
installing to /tmp/th798/64531145/R-4.2.3/392/library/00LOCK-FIESTAutils/00new/FIESTAutils/libs
** R
** data
*** moving datasets to lazyload DB
** inst
** byte-compile and prepare package for lazy loading
Error in dyn.load(file, DLLpath = DLLpath, ...) : 
  unable to load shared object '/projects/genomic-ml/R/R-4.2.3/library/sf/libs/sf.so':
  libgdal.so.28: cannot open shared object file: No such file or directory
Calls: <Anonymous> ... namespaceImport -> loadNamespace -> library.dynam -> dyn.load
Execution halted
ERROR: lazy loading failed for package 'FIESTAutils'
* removing '/tmp/th798/64531145/R-4.2.3/392/library/FIESTAutils'

This is problematic for two reasons.

• this failure was not reported at all on the summary web page, which makes the user think there is no problem, but there could be. it should be reported as a configuration failure.
• the configuration failure should be fixed so we can see if there is any problem with FIESTA.

old build April 5 has

* checking re-building of vignette outputs ... OK (data.table from CRAN)
vs
* checking re-building of vignette outputs ... ERROR (data.table from github master)
Error(s) in re-building vignettes:
--- re-building 'FIESTA_manual_mod_est.Rmd' using rmarkdown
--- finished re-building 'FIESTA_manual_mod_est.Rmd'

--- re-building 'FIESTA_manual_mod_pop.Rmd' using rmarkdown
--- finished re-building 'FIESTA_manual_mod_pop.Rmd'

--- re-building 'FIESTA_tutorial_DB.Rmd' using rmarkdown
--- finished re-building 'FIESTA_tutorial_DB.Rmd'

--- re-building 'FIESTA_tutorial_GB.Rmd' using rmarkdown
Warning: call dbDisconnect() when finished working with a connection
--- finished re-building 'FIESTA_tutorial_GB.Rmd'

--- re-building 'FIESTA_tutorial_MA.Rmd' using rmarkdown
Quitting from lines 218-224 (FIESTA_tutorial_MA.Rmd) 
Error: processing vignette 'FIESTA_tutorial_MA.Rmd' failed with diagnostics:
Column or expression 1 of 'by' or 'keyby' is type 'NULL' which is not currently supported. If you have a compelling use case, please add it to https://github.com/Rdatatable/data.table/issues/1597. As a workaround, consider converting the column to a supported type, e.g. by=sapply(list_col, toString), whilst taking care to maintain distinctness in the process.
--- failed re-building 'FIESTA_tutorial_MA.Rmd'

--- re-building 'FIESTA_tutorial_PB.Rmd' using rmarkdown
--- finished re-building 'FIESTA_tutorial_PB.Rmd'

--- re-building 'FIESTA_tutorial_SA.Rmd' using rmarkdown
Quitting from lines 219-223 (FIESTA_tutorial_SA.Rmd) 
Error: processing vignette 'FIESTA_tutorial_SA.Rmd' failed with diagnostics:
Incompatible join types: x.CONDID (integer) and i.DOMAIN (character)
--- failed re-building 'FIESTA_tutorial_SA.Rmd'

--- re-building 'FIESTA_tutorial_dat.Rmd' using rmarkdown
--- finished re-building 'FIESTA_tutorial_dat.Rmd'

--- re-building 'FIESTA_tutorial_sp.Rmd' using rmarkdown
--- finished re-building 'FIESTA_tutorial_sp.Rmd'

SUMMARY: processing the following files failed:
  'FIESTA_tutorial_MA.Rmd' 'FIESTA_tutorial_SA.Rmd'

Error: Vignette re-building failed.
Execution halted

current CRAN build has

Result: ERROR
    Error(s) in re-building vignettes:
     ...
    --- re-building ‘FIESTA_manual_mod_est.Rmd’ using rmarkdown
    --- finished re-building ‘FIESTA_manual_mod_est.Rmd’

    --- re-building ‘FIESTA_manual_mod_pop.Rmd’ using rmarkdown
    --- finished re-building ‘FIESTA_manual_mod_pop.Rmd’

    --- re-building ‘FIESTA_tutorial_DB.Rmd’ using rmarkdown
    --- finished re-building ‘FIESTA_tutorial_DB.Rmd’

    --- re-building ‘FIESTA_tutorial_GB.Rmd’ using rmarkdown
    Warning: call dbDisconnect() when finished working with a connection
    --- finished re-building ‘FIESTA_tutorial_GB.Rmd’

    --- re-building ‘FIESTA_tutorial_MA.Rmd’ using rmarkdown
    Quitting from lines 111-136 (FIESTA_tutorial_MA.Rmd)
    Error: processing vignette ‘FIESTA_tutorial_MA.Rmd’ failed with diagnostics:
    Cannot create a RasterLayer object from this file.
    --- failed re-building ‘FIESTA_tutorial_MA.Rmd’

    --- re-building ‘FIESTA_tutorial_PB.Rmd’ using rmarkdown
    --- finished re-building ‘FIESTA_tutorial_PB.Rmd’

    --- re-building ‘FIESTA_tutorial_SA.Rmd’ using rmarkdown
    Quitting from lines 105-135 (FIESTA_tutorial_SA.Rmd)
    Error: processing vignette ‘FIESTA_tutorial_SA.Rmd’ failed with diagnostics:
    Cannot create a RasterLayer object from this file.
    --- failed re-building ‘FIESTA_tutorial_SA.Rmd’

    --- re-building ‘FIESTA_tutorial_dat.Rmd’ using rmarkdown
    --- finished re-building ‘FIESTA_tutorial_dat.Rmd’

    --- re-building ‘FIESTA_tutorial_sp.Rmd’ using rmarkdown
    --- finished re-building ‘FIESTA_tutorial_sp.Rmd’

    SUMMARY: processing the following files failed:
     ‘FIESTA_tutorial_MA.Rmd’ ‘FIESTA_tutorial_SA.Rmd’

    Error: Vignette re-building failed.

current monsoon build has the same (data.table issue masked by other issue which is also on CRAN, nothing to fix in our check system).

[tdhock]

this is affecting 1.15.0 release https://github.com/Rdatatable/data.table/issues/5823

* installing *source* package 'tcltk2' ...
** package 'tcltk2' successfully unpacked and MD5 sums checked
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
Error: package or namespace load failed for 'tcltk':
 .onLoad failed in loadNamespace() for 'tcltk', details:
  call: fun(libname, pkgname)
  error: Tcl/Tk support is not available on this system
In addition: Warning message:
S3 methods 'as.character.tclObj', 'as.character.tclVar', 'as.double.tclObj', 'as.integer.tclObj', 'as.logical.tclObj', 'as.raw.tclObj', 'print.tclObj', '[[.tclArray', '[[<-.tclArray', '$.tclArray', '$<-.tclArray', 'names.tclArray', 'names<-.tclArray', 'length.tclArray', 'length<-.tclArray', 'tclObj.tclVar', 'tclObj<-.tclVar', 'tclvalue.default', 'tclvalue.tclObj', 'tclvalue.tclVar', 'tclvalue<-.default', 'tclvalue<-.tclVar', 'close.tkProgressBar' were declared in NAMESPACE but not found 
Execution halted
ERROR: lazy loading failed for package 'tcltk2'
* removing '/tmp/th798/7253742/R-devel/1045/library/tcltk2'
...
ERROR: dependency 'tcltk2' is not available for package 'Rcmdr'
* removing '/tmp/th798/7253742/R-devel/1045/library/Rcmdr'
...
* installing *source* package 'tkrplot' ...
** package 'tkrplot' successfully unpacked and MD5 sums checked
** using staged installation
./configure: line 1747: R_TCLTK: command not found
configure: creating ./config.status
config.status: creating src/Makevars
** libs
using C compiler: 'gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-20)'
gcc -I"/home/th798/R/R-devel/include" -DNDEBUG   -I/home/th798/.conda/envs/emacs1/include -I/home/th798/include    -fpic  -g -O2  -c tcltkimg.c -o tcltkimg.o
gcc -shared -L/home/th798/.conda/envs/emacs1/lib -Wl,-rpath=/home/th798/.conda/envs/emacs1/lib -L/home/th798/lib -Wl,-rpath=/home/th798/lib -L/home/th798/lib64 -Wl,-rpath=/home/th798/lib64 -o tkrplot.so tcltkimg.o
installing to /tmp/th798/7253742/R-devel/1045/library/00LOCK-tkrplot/00new/tkrplot/libs
** R
** byte-compile and prepare package for lazy loading
Error: package or namespace load failed for 'tcltk':
 .onLoad failed in loadNamespace() for 'tcltk', details:
  call: fun(libname, pkgname)
  error: Tcl/Tk support is not available on this system
In addition: Warning message:
S3 methods 'as.character.tclObj', 'as.character.tclVar', 'as.double.tclObj', 'as.integer.tclObj', 'as.logical.tclObj', 'as.raw.tclObj', 'print.tclObj', '[[.tclArray', '[[<-.tclArray', '$.tclArray', '$<-.tclArray', 'names.tclArray', 'names<-.tclArray', 'length.tclArray', 'length<-.tclArray', 'tclObj.tclVar', 'tclObj<-.tclVar', 'tclvalue.default', 'tclvalue.tclObj', 'tclvalue.tclVar', 'tclvalue<-.default', 'tclvalue<-.tclVar', 'close.tkProgressBar' were declared in NAMESPACE but not found 
Execution halted
ERROR: lazy loading failed for package 'tkrplot'
* removing '/tmp/th798/7253742/R-devel/1045/library/tkrplot'

below we see the following check result, using both versions of data.table.

* using log directory '/tmp/th798/7253742/R-devel/1045/RcmdrPlugin.RiskDemo.Rcheck'
* using R Under development (unstable) (2024-01-14 r85805)
* using platform: x86_64-pc-linux-gnu
* R was compiled by
    gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-20)
    GNU Fortran (GCC) 8.5.0 20210514 (Red Hat 8.5.0-20)
* running under: Red Hat Enterprise Linux 8.9 (Ootpa)
* using session charset: ASCII
* checking for file 'RcmdrPlugin.RiskDemo/DESCRIPTION' ... OK
* checking extension type ... Package
* this is package 'RcmdrPlugin.RiskDemo' version '3.1'
* checking package namespace information ... OK
* checking package dependencies ... ERROR
Package required but not available: 'Rcmdr'

Package suggested but not available for checking: 'tkrplot'

See section 'The DESCRIPTION file' in the 'Writing R Extensions'
manual.
* DONE

Status: 1 ERROR
See
  '/tmp/th798/7253742/R-devel/1045/RcmdrPlugin.RiskDemo.Rcheck/00check.log'
for details.

This is a false negative (there is a significant difference here, but it is not being reported).

[tdhock]

Looks like the common thread is the following, which could be parsed and reported, just like "configuration failed"

ERROR: lazy loading failed for package 'tkrplot'

grep.config.fail <- paste(
  "grep 'configuration failed for package'",
  log.file.glob)
config.fail.vec <- suppressWarnings({
  system(grep.config.fail, intern=TRUE)
})

[tdhock]

using new code https://github.com/tdhock/data.table-revdeps/commit/01787e8cb7e386ab8342e037997ace803085050f we get output in the report as below