recurrent TIMEOUTs hanging on re-building vignettes

tdhock commented 9 months ago

ctmm

Time to install revdep:
> print(install.time)
   user  system elapsed 
167.241   7.738 216.494 
...
Time to install revdep:
> print(install.time)
   user  system elapsed 
141.970   7.990 174.316 
...
* checking data for ASCII and uncompressed saves ... OK
* checking installed files from 'inst/doc' ... OK
* checking files in 'vignettes' ... OK
* checking examples ... OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in 'inst/doc' ... OK
* checking running R code from vignettes ...
  'akde.Rmd' using 'UTF-8'... OK
  'error.Rmd' using 'UTF-8'... OK
  'interactions.Rmd' using 'UTF-8'... OK
  'periodogram.Rmd' using 'UTF-8'... OK
  'variogram.Rmd' using 'UTF-8'... OK
 NONE
* checking re-building of vignette outputs ...slurmstepd: error: *** JOB 7594840 ON cn5 CANCELLED AT 2024-02-12T16:02:13 DUE TO TIME LIMIT ***

tdhock commented 9 months ago

haldensify

Time to install revdep:
> print(install.time)
   user  system elapsed 
 56.002   5.425  97.850 
...
* checking examples ... OK
* checking for unstated dependencies in 'tests' ... OK
* checking tests ...
  Running 'testthat.R'
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... OK
* checking re-building of vignette outputs ...slurmstepd: error: *** JOB 7595134 ON cn76 CANCELLED AT 2024-02-12T16:02:13 DUE TO TIME LIMIT ***

tdhock commented 9 months ago

maybe related: FAILED status has the following at end of log:

/usr/bin/xvfb-run: line 186: kill: (1604573) - No such process

tdhock commented 9 months ago

also some lazy loading failed for packages which depends on x11, such as RcmdrPlugin.WorldFlora

* installing *source* package 'tcltk2' ...
** package 'tcltk2' successfully unpacked and MD5 sums checked
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
X connection to :99 broken (explicit kill or server shutdown).
ERROR: lazy loading failed for package 'tcltk2'
* removing '/tmp/th798/7605806/R-devel/1048/library/tcltk2'
...
/usr/bin/xvfb-run: line 186: kill: (842586) - No such process

tdhock commented 9 months ago

maybe x11 is being killed by another task on the same machine?

tdhock commented 9 months ago

https://manpages.ubuntu.com/manpages/xenial/man1/xvfb-run.1.html says

-f file, --auth-file=file
              Store X authentication data in file.  By  default,  a  temporary  directory  called
              xvfb-run.PID  (where  PID  is  the process ID of xvfb-run itself) is created in the
              directory specified by the environment variable TMPDIR (or [/tmp](file:///tmp) if that variable is
              null  or  unset),  and  the  [tempfile](https://manpages.ubuntu.com/manpages/xenial/man1/tempfile.1.html)(1)  command  is used to create a file in that
              temporary directory called Xauthority.

so maybe worth trying specifying this dir

tdhock commented 9 months ago

help on monsoon says something different:

th798@cn61:~/genomic-ml/data.table-revdeps(master*)$ xvfb-run --help
Usage: xvfb-run [OPTION ...] COMMAND
Run COMMAND (usually an X client) in a virtual X server environment.
Options:
-a        --auto-servernum          try to get a free server number, starting at
                                    --server-num (deprecated, use --auto-display
                                    instead)
-d        --auto-display            use the X server to find a display number
                                    automatically
-e FILE   --error-file=FILE         file used to store xauth errors and Xvfb
                                    output (default: /dev/null)
-f FILE   --auth-file=FILE          file used to store auth cookie
                                    (default: ./.Xauthority)
-h        --help                    display this usage message and exit
-n NUM    --server-num=NUM          server number to use (default: 99)

what are allowable server numbers? why does it say auth file option is unrecognized below?

th798@cn61:~/genomic-ml/data.table-revdeps(master*)$ xvfb-run --auth-file=$TMPDIR/100/Xauthority R
xvfb-run: unrecognized option '--auth-file=/tmp/th798/7614885/100/Xauthority'

tdhock commented 9 months ago

https://stackoverflow.com/questions/12153552/how-high-do-x11-display-numbers-go

tdhock commented 9 months ago

could it be the display numbers are conflicting?

th798@cn61:~$ xvfb-run Rscript -e 'dir("/tmp/.X11-unix")'
Loading required namespace: BiocManager
[1] "X99"
th798@cn61:~$ xvfb-run -d Rscript -e 'dir("/tmp/.X11-unix")'
Loading required namespace: BiocManager
[1] "X0"

use -d option to get another one?

th798@cn61:~$ xvfb-run -d R

R version 4.3.1 (2023-06-16) -- "Beagle Scouts"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

Loading required namespace: BiocManager
> dir("/tmp/.X11-unix/")
[1] "X0"
> system("xvfb-run -d Rscript -e 'dir(\"/tmp/.X11-unix\")'")
Loading required namespace: BiocManager
[1] "X0" "X1"
>

Looks like using -d option above may fix this issue, and not using it below causes it:

th798@cn61:~$ xvfb-run R

R version 4.3.1 (2023-06-16) -- "Beagle Scouts"
Copyright (C) 2023 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

Loading required namespace: BiocManager
> dir("/tmp/.X11-unix/")
[1] "X99"
> system("xvfb-run Rscript -e 'dir(\"/tmp/.X11-unix\")'")
Loading required namespace: BiocManager
[1] "X99"
/usr/bin/xvfb-run: line 186: kill: (2584678) - No such process
>

tdhock commented 9 months ago

try to fix with https://github.com/tdhock/data.table-revdeps/commit/3eb6c80e54d1e1245a3f439077e2d185a9dc8d94 which actually fixed most of the FAILED, but did not fix recurrent TIMEOUT

tdhock commented 9 months ago

in ctmm there are 4 checking re-building vignette outputs. First 3 work fine.

R-devel dt-release OK
R-devel dt-master OK
R-release 4.3.2 dt-release OK
R-release dt-master HANG

in haldensify there are only 2 checks, first 1 works fine

R-devel dt-release OK
R-devel dt-master HANG

WHY?

tdhock commented 9 months ago

adding time to check seems to indicate that the checks are just taking a long time. https://cloud.r-project.org/web/checks/check_results_ctmm.html says CRAn skips vignettes.

tdhock commented 9 months ago

--no-vignettes means (from R CMD check --help) --no-vignettes do not run R code in vignettes nor build outputs

tdhock commented 9 months ago

haldensify does not have any gh actions runs https://github.com/nhejazi/haldensify/actions ctmm does not run checks on actions https://github.com/ctmm-initiative/ctmm/actions

tdhock commented 9 months ago

haldensify takes over 7 hours for one check

[1] "2024-02-17 03:34:28 MST"
* using log directory '/tmp/th798/7667922/R-devel/527/haldensify.Rcheck'
* using R Under development (unstable) (2024-02-16 r85931)
* using platform: x86_64-pc-linux-gnu
* R was compiled by
    gcc (Spack GCC) 12.2.0
    GNU Fortran (Spack GCC) 12.2.0
* running under: Red Hat Enterprise Linux 8.9 (Ootpa)
...
* checking package vignettes ... OK
* checking re-building of vignette outputs ... OK
* checking PDF version of manual ... OK
* DONE

Status: OK

[1] "2024-02-17 10:53:38 MST"

...

[1] "2024-02-17 10:53:58 MST"
* using log directory '/tmp/th798/7667922/R-devel/527/haldensify.Rcheck'
* using R Under development (unstable) (2024-02-16 r85931)
* using platform: x86_64-pc-linux-gnu
* R was compiled by
    gcc (Spack GCC) 12.2.0
    GNU Fortran (Spack GCC) 12.2.0
* running under: Red Hat Enterprise Linux 8.9 (Ootpa)
...
* checking tests ...
  Running 'testthat.R'
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... OK
* checking re-building of vignette outputs ...slurmstepd: error: *** JOB 7667922 ON cn76 CANCELLED AT 2024-02-17T12:32:08 DUE TO TIME LIMIT ***

ctmm takes over 3 hours, see below

[1] "2024-02-17 03:35:53 MST"
* using log directory '/tmp/th798/7667628/R-devel/233/ctmm.Rcheck'
* using R Under development (unstable) (2024-02-16 r85931)
* using platform: x86_64-pc-linux-gnu
* R was compiled by
    gcc (Spack GCC) 12.2.0
    GNU Fortran (Spack GCC) 12.2.0
* running under: Red Hat Enterprise Linux 8.9 (Ootpa)
...
* checking package vignettes ... OK
* checking re-building of vignette outputs ... OK
* checking PDF version of manual ... OK
* DONE

Status: 1 WARNING
See
  '/tmp/th798/7667628/R-devel/233/ctmm.Rcheck/00check.log'
for details.

[1] "2024-02-17 06:49:05 MST"

...

[1] "2024-02-17 11:25:32 MST"
* using log directory '/tmp/th798/7667628/R-4.3.2/233/ctmm.Rcheck'
* using R version 4.3.2 (2023-10-31)
* using platform: x86_64-pc-linux-gnu (64-bit)
* R was compiled by
    gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-20)
    GNU Fortran (GCC) 8.5.0 20210514 (Red Hat 8.5.0-20)
* running under: Red Hat Enterprise Linux 8.9 (Ootpa)
...
* checking running R code from vignettes ...
  'akde.Rmd' using 'UTF-8'... OK
  'error.Rmd' using 'UTF-8'... OK
  'interactions.Rmd' using 'UTF-8'... OK
  'periodogram.Rmd' using 'UTF-8'... OK
  'variogram.Rmd' using 'UTF-8'... OK
 NONE
* checking re-building of vignette outputs ...slurmstepd: error: *** JOB 7667628 ON cn12 CANCELLED AT 2024-02-17T12:32:08 DUE TO TIME LIMIT ***

tdhock commented 8 months ago

same for aopdata

[1] "2024-02-28 00:54:04 MST"
* using log directory '/tmp/th798/7844182/R-release/38/aopdata.Rcheck'
* using R version 4.3.2 (2023-10-31)
* using platform: x86_64-pc-linux-gnu (64-bit)
* R was compiled by
    gcc (Spack GCC) 12.2.0
    GNU Fortran (Spack GCC) 12.2.0
* running under: Red Hat Enterprise Linux 8.9 (Ootpa)
* using session charset: ASCII
* checking for file 'aopdata/DESCRIPTION' ... OK
* this is package 'aopdata' version '1.0.3'
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package 'aopdata' can be installed ... OK
* checking installed package size ... OK
* checking package directory ... OK
* checking 'build' directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking installed files from 'inst/doc' ... OK
* checking files in 'vignettes' ... OK
* checking examples ... OK
* checking for unstated dependencies in 'tests' ... OK
* checking tests ...
  Running 'testthat.R'
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in 'inst/doc' ... OK
* checking running R code from vignettes ...
  'access_inequality.Rmd' using 'UTF-8'... OK
  'access_maps.Rmd' using 'UTF-8'... OK
  'data_dic_en.Rmd' using 'UTF-8'... OK
  'data_dic_pt.Rmd' using 'UTF-8'... OK
  'intro_to_aopdata.Rmd' using 'UTF-8'... OK
  'landuse_maps.Rmd' using 'UTF-8'... OK
  'population_maps.Rmd' using 'UTF-8'... OK
 NONE
* checking re-building of vignette outputs ...slurmstepd: error: *** JOB 7844182 ON cn26 CANCELLED AT 2024-02-28T09:46:06 DUE TO TIME LIMIT ***

tdhock commented 8 months ago

same for hal9001


[1] "2024-02-28 05:47:57 MST"
* using log directory '/tmp/th798/7844677/R-devel/529/hal9001.Rcheck'
* using R Under development (unstable) (2024-02-27 r85995)
* using platform: x86_64-pc-linux-gnu
* R was compiled by
    gcc (Spack GCC) 12.2.0
    GNU Fortran (Spack GCC) 12.2.0
* running under: Red Hat Enterprise Linux 8.9 (Ootpa)
* using session charset: ASCII
* checking for file 'hal9001/DESCRIPTION' ... OK
* this is package 'hal9001' version '0.4.6'
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package 'hal9001' can be installed ... OK
* used C++ compiler: 'g++ (Spack GCC) 12.2.0'
* checking installed package size ... OK
* checking package directory ... OK
* checking 'build' directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking whether startup messages can be suppressed ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking contents of 'data' directory ... OK
* checking data for non-ASCII characters ... OK
* checking LazyData ... OK
* checking data for ASCII and uncompressed saves ... OK
* checking line endings in C/C++/Fortran sources/headers ... OK
* checking line endings in Makefiles ... OK
* checking compilation flags in Makevars ... OK
* checking for GNU extensions in Makefiles ... OK
* checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK
* checking use of PKG_*FLAGS in Makefiles ... OK
* checking compiled code ... OK
* checking installed files from 'inst/doc' ... OK
* checking files in 'vignettes' ... OK
* checking examples ... OK
Examples with CPU (user + system) or elapsed time > 5s
           user system elapsed
fit_hal 341.396  0.033  342.75
* checking for unstated dependencies in 'tests' ... OK
* checking tests ...
  Running 'testthat.R'
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... OK
* checking re-building of vignette outputs ...slurmstepd: error: *** JOB 7844677 ON cn18 CANCELLED AT 2024-02-28T09:51:36 DUE TO TIME LIMIT ***

tdhock commented 8 months ago

checking on windows machine is not slow, so this is a monsoon issue

th798@cmp2986 MINGW64 ~/R
$ R CMD check --timings hal9001_0.4.6.tar.gz
Loading required namespace: BiocManager
* using log directory 'C:/Users/th798/R/hal9001.Rcheck'
* using R Under development (unstable) (2024-01-23 r85822 ucrt)
* using platform: x86_64-w64-mingw32
* R was compiled by
    gcc.exe (GCC) 12.3.0
    GNU Fortran (GCC) 12.3.0
* running under: Windows 10 x64 (build 19045)
* using session charset: UTF-8
* checking for file 'hal9001/DESCRIPTION' ... OK
* this is package 'hal9001' version '0.4.6'
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking whether package 'hal9001' can be installed ... OK
* used C++ compiler: 'G__~1.EXE (GCC) 12.3.0'
* checking installed package size ... OK
* checking package directory ... OK
* checking 'build' directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking whether startup messages can be suppressed ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking contents of 'data' directory ... OK
* checking data for non-ASCII characters ... OK
* checking LazyData ... OK
* checking data for ASCII and uncompressed saves ... OK
* checking line endings in C/C++/Fortran sources/headers ... OK
* checking line endings in Makefiles ... OK
* checking compilation flags in Makevars ... OK
* checking for GNU extensions in Makefiles ... OK
* checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK
* checking use of PKG_*FLAGS in Makefiles ... OK
* checking compiled code ... OK
* checking installed files from 'inst/doc' ... OK
* checking files in 'vignettes' ... OK
* checking examples ... OK
* checking for unstated dependencies in 'tests' ... OK
* checking tests ...
  Running 'testthat.R'
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... OK
* checking re-building of vignette outputs ... OK
* checking PDF version of manual ... OK
* DONE

Status: OK

(base) ]0;MINGW64:/c/Users/th798/R
th798@cmp2986 MINGW64 ~/R
$ exit

Process shell finished

tdhock / data.table-revdeps

recurrent TIMEOUTs hanging on re-building vignettes #17