Closed mattdowle closed 4 years ago
Hello,
Thanks @mattdowle for reproducing the error with gctorture. If I am reading the log-file right, the operation causing the GCC-ASAN error should be this one.
an.MCMC <- an.MCMC[,
list(
PostMean = mean(unlist(Mean)), # For optimization, replaced by .External(Cfastmean, unlist(Mean), FALSE)???
HPD = list(colMeans(do.call(rbind, HPD))),
HPD50 = list(colMeans(do.call(rbind, HPD50))),
start = start(mcs),
end = end(mcs),
thin = thin(mcs),
ESS = sum(unlist(ESS)),
G.R. = if(length(mcs)>1) {
gelman.diag(mcs, autoburnin=FALSE)$psrf[1]
} else {
as.double(NA)
},
nChains = length(mcs),
mcmc = mcmc.list(mcmc(do.call(rbind, mcs)))),
by=list(stat, samplePriorMCMC)]
The above code is located in the function summary.POUMM, file summaryPOUMM.R in the POUMM R-package.
I tried to create a minimal example (stripped of any POUMM-specific stuff) that should reproduce the error.
Please, do this first:
Hope this will reproduce the error faster. If it doesn't, then, probably something wrong happens earlier in the process but only manifests at this line of code. Let's see...
Thanks, @venelin .
I used v1.13.0 to reproduce. Rerunning with data.table dev as of now, under torture, the full POUMM vignette code completes ok. I'm thinking #4655 fixed this one then since dogroups.c
, list columns, protection/copies are common, and #4655 was a much cleaner and direct root cause fix in this area. However, it would be nice to know a bit more as to why, so I'll keep going with the goal in mind to explain why #4655 fixes it. It's possible that gctorture2(step=100)
isn't always sufficient, and this run passed by chance, so I'll do a run with gctorture2(step=50)
too.
The run with gctorture2(step=50)
passed ok with data.table-dev and took 10 hours 8 mins. Good.
Running again with data.table 1.13.0 to make sure it fails ...
Good it failed in 1hr 7mins with output below. This time with line numbers in dogroups.c
which is good. In the meantime I had realized my ~/.R/Makevars did not include -g
so when I had installed into R-devel-strict before, it wasn't compiling with -g
and hence, looking more closely at original output pasted above, I was only getting datatable.so
references there without a line number.
Now just to explain it and make a test.
$ Rdevel-strict-gcc
R Under development (unstable) (2020-07-14 r78854) -- "Unsuffered Consequences"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> source("~/tmp/POUMM.txt", echo=TRUE)
> cat("***** Starting POUMM.txt at", format(Sys.time()), "\n")
***** Starting POUMM.txt at 2020-10-11 09:57:07
> ## ----setup, include = FALSE----------------------------------------------
> # Make results reproducible
> set.seed(1)
> knitr::opts_chunk$set(cache = FALSE)
> options(digits = 4)
> library(POUMM)
Loading required package: Rcpp
date
> useCachedResults <- TRUE
> ## ----install-CRAN-packages, eval=FALSE-----------------------------------
> # install.packages("data.table")
> # install.packages("ggplot2")
> # .... [TRUNCATED]
Attaching package: ‘ggplot2’
The following object is masked from ‘package:POUMM’:
alpha
> library(data.table)
data.table 1.13.0 using 1 threads (see ?getDTthreads). Latest news: r-datatable.com
**********
This installation of data.table has not detected OpenMP support. It should still work but in single-threaded mode.
This is Linux. This warning should not normally occur on Windows or Linux where OpenMP is turned on by data.table's configure script by passing -fopenmp to the compiler. If you see this warning on Windows or Linux, please file a GitHub issue.
**********
> library(lmtest)
Loading required package: zoo
Attaching package: ‘zoo’
The following objects are masked from ‘package:base’:
as.Date, as.Date.numeric
> library(ape)
> ## ---- eval=FALSE, echo=TRUE----------------------------------------------
> # devtools::install_github(repo="venelin/POUMM")
>
> ## ---- eval=FA .... [TRUNCATED]
> g0 <- 0
> alpha <- .5
> theta <- 2
> sigma <- 0.2
> sigmae <- 0.2
> ## ---- include=FALSE, eval=useCachedResults-------------------------------
> data(vignetteCachedResults)
> list2env(vignetteCachedResults, globalenv())
<environment: R_GlobalEnv>
> ## ---- echo=FALSE, fig.height=5, fig.width=7, fig.cap="Dashed black and magenta lines denote the deterministic trend towards the long-term mean $\\ ..." ... [TRUNCATED]
> t <- seq(0, 6, by = tStep)
> plot(t, rTrajectoryOU(g0, tStep, alpha, theta, sigma, length(t)), type = 'l', ylab = expression(bar(z)), ylim = c(0, 4))
> #plot(t, rTrajectoryOU(g0, tStep, alpha, theta, sigma, length(t)), type = 'l', main = "Random OU trajectories", ylab = "g", ylim = c(0, 4))
> lines( .... [TRUNCATED]
> lines(t, rTrajectoryOU(g0, tStep, alpha*2, theta, sigma, length(t)), col = "magenta")
> lines(t, rTrajectoryOU(g0, tStep, alpha*2, theta, 0, length(t)), lty = 2, col = "magenta")
> lines(t, rTrajectoryOU(g0, tStep, alpha, theta, sigma*2, length(t)), col = "blue")
> abline(h=theta, lty = 3, col = "darkgrey")
> legend("topleft",
+ legend = c(expression(list(alpha == .5, sigma^2 == 0.04)),
+ expression(list(alpha == .5, sigma^2 == 0 .... [TRUNCATED]
> ## ----simulate-tree, results="hide", eval=!useCachedResults---------------
> # # Number of tips
> # tree <- rtree(N)
> #
> # plot(tree, show.t .... [TRUNCATED]
> data <- data[, group := cut(t, breaks = 5, include.lowest = TRUE)]
> ggplot(data = data, aes(x = t, y = z, group = group)) +
+ geom_violin(aes(col = group)) + geom_point(aes(col = group), size=.5)
> ## ----MaintainCache, echo=FALSE, warning=FALSE, results="hide", message=FALSE, eval=TRUE----
> if(!useCachedResults) {
+ # Perform the heavy fits .... [TRUNCATED]
> ## ----RestorePruneInfo, echo=FALSE, warning=FALSE, results="hide", message=FALSE, eval=TRUE----
> # restore the pruneInfo since it is needed afterw .... [TRUNCATED]
> ## ----fitPOUMM-1, results="hide", message=FALSE, warning=FALSE, eval=FALSE----
> # fitPOUMM <- POUMM(z[1:N], tree)
>
> ## ---- fig.height=5.4, fi .... [TRUNCATED]
> plotList$traceplot
> ## ---- fig.height=5.4, fig.show="hold", fig.width=7.2, warning=FALSE, fig.cap="MCMC univariate density plots. Black dots on the x-axis indicate the ..." ... [TRUNCATED]
> cat("***** Starting gctorture2 at", format(Sys.time()), "\n")
***** Starting gctorture2 at 2020-10-11 09:57:43
> gctorture2(step=100)
[1] 0
> options(datatable.verbose=TRUE)
> ## ---- warning=FALSE, eval=TRUE-------------------------------------------
> summary(fitPOUMM)
Zero-copy coerce when assigning 'logical' to 'double' column 2 named 'MLE'.
Detected that j uses these columns: <none>
Assigning to all 16 rows
RHS_list_of_columns == false
Detected that j uses these columns: <none>
Assigning to all 48 rows
RHS_list_of_columns == false
Direct plonk of unnamed RHS, no copy. NAMED==1, MAYBE_SHARED==0
Detected that j uses these columns: Mean,HPD,HPD50,start,mcs,end,ESS
Finding groups using forderv ... forder.c received 48 rows and 2 columns
0.841s elapsed (0.840s cpu)
Finding group sizes from the positions (can be avoided to save RAM) ... 0.166s elapsed (0.166s cpu)
lapply optimization is on, j unchanged as 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
GForce is on, left j unchanged
Old mean optimization changed j from 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) { gelman.diag(mcs, autoburnin = FALSE)$psrf[1] } else { as.double(NA) }, length(mcs), mcmc.list(mcmc(do.call(rbind, mcs))))' to 'list(.External(Cfastmean, unlist(Mean), FALSE), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
Making each group and running j (GForce FALSE) ... =================================================================
==437857==ERROR: AddressSanitizer: heap-use-after-free on address 0x6170004b8303 at pc 0x5575a26bf022 bp 0x7fff497b97e0 sp 0x7fff497b97d0
READ of size 1 at 0x6170004b8303 thread T0
#0 0x5575a26bf021 in RunGenCollect /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:1787
#1 0x5575a26bf021 in R_gc_internal /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:3125
#2 0x5575a26d35e5 in Rf_cons /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:2405
#3 0x5575a24b2087 in duplicate_list /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:264
#4 0x5575a24b2087 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:312
#5 0x5575a24b2b50 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:352
#6 0x5575a24b1ff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:206
#7 0x5575a24b1ff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:204
#8 0x5575a24b1ff9 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:338
#9 0x7f77404c0631 in copyAsPlain /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/utils.c:246
#10 0x7f774038fa5f in memrecycle /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/assign.c:721
#11 0x7f77403c52a4 in dogroups /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/dogroups.c:166
#12 0x5575a2447875 in R_doDotCall /home/mdowle/build/R-devel-strict-gcc/src/main/dotcode.c:669
#13 0x5575a259c818 in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7670
#14 0x5575a25d7787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#15 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#16 0x5575a25dce61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#17 0x5575a26edf61 in dispatchMethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:436
#18 0x5575a26ee6a5 in Rf_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:476
#19 0x5575a256eabb in tryDispatch /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:5362
#20 0x5575a258825b in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7260
#21 0x5575a25d7787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#22 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#23 0x5575a25dce61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#24 0x5575a26edf61 in dispatchMethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:436
#25 0x5575a26ee6a5 in Rf_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:476
#26 0x5575a26eec4d in do_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:565
#27 0x5575a25d7b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#28 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#29 0x5575a25dce61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#30 0x5575a25d78fb in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:846
#31 0x5575a25e3879 in do_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:3358
#32 0x5575a26e7eb0 in do_internal /home/mdowle/build/R-devel-strict-gcc/src/main/names.c:1385
#33 0x5575a25d7b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#34 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#35 0x5575a25dce61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#36 0x5575a25d78fb in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:846
#37 0x5575a25d8c2a in forcePromise /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:551
#38 0x5575a25d836c in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:746
#39 0x5575a25e412b in do_withVisible /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:3387
#40 0x5575a26e7eb0 in do_internal /home/mdowle/build/R-devel-strict-gcc/src/main/names.c:1385
#41 0x5575a25d7b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#42 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#43 0x5575a25dce61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#44 0x5575a25d78fb in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:846
#45 0x5575a25e098e in do_set /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:2960
#46 0x5575a25d7b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#47 0x5575a25de496 in do_begin /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:2508
#48 0x5575a25d7b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#49 0x5575a25d7b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#50 0x5575a25de496 in do_begin /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:2508
#51 0x5575a25d7b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#52 0x5575a25e6cd2 in do_for /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:2390
#53 0x5575a25d7b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#54 0x5575a25de496 in do_begin /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:2508
#55 0x5575a25d7b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#56 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#57 0x5575a25dce61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#58 0x5575a25d78fb in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:846
#59 0x5575a2665c35 in Rf_ReplIteration /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:264
#60 0x5575a2666620 in R_ReplConsole /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:314
#61 0x5575a26667b6 in run_Rmainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1113
#62 0x5575a2666826 in Rf_mainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1120
#63 0x5575a22bae41 in main /home/mdowle/build/R-devel-strict-gcc/src/main/Rmain.c:29
#64 0x7f774cfd20b2 in __libc_start_main (/usr/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
#65 0x5575a22bd67d in _start (/home/mdowle/build/R-devel-strict-gcc/bin/exec/R+0x7e367d)
0x6170004b8303 is located 3 bytes inside of 768-byte region [0x6170004b8300,0x6170004b8600)
freed by thread T0 here:
#0 0x7f774e3dd7cf in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
#1 0x5575a26b382e in ReleaseLargeFreeVectors /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:1114
#2 0x5575a26b382e in RunGenCollect /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:1896
#3 0x5575a26b382e in R_gc_internal /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:3125
#4 0x5575a26d35e5 in Rf_cons /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:2405
#5 0x5575a24b2087 in duplicate_list /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:264
#6 0x5575a24b2087 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:312
#7 0x5575a24b2b50 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:352
#8 0x5575a24b1ff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:206
#9 0x5575a24b1ff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:204
#10 0x5575a24b1ff9 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:338
#11 0x7f77404c0631 in copyAsPlain /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/utils.c:246
#12 0x7f774038fa5f in memrecycle /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/assign.c:721
#13 0x7f77403c52a4 in dogroups /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/dogroups.c:166
#14 0x5575a2447875 in R_doDotCall /home/mdowle/build/R-devel-strict-gcc/src/main/dotcode.c:669
#15 0x5575a259c818 in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7670
#16 0x5575a25d7787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#17 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#18 0x5575a25dce61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#19 0x5575a26edf61 in dispatchMethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:436
#20 0x5575a26ee6a5 in Rf_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:476
#21 0x5575a256eabb in tryDispatch /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:5362
#22 0x5575a258825b in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7260
#23 0x5575a25d7787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#24 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#25 0x5575a25dce61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#26 0x5575a26edf61 in dispatchMethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:436
#27 0x5575a26ee6a5 in Rf_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:476
#28 0x5575a26eec4d in do_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:565
#29 0x5575a25d7b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#30 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#31 0x5575a25dce61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#32 0x5575a25d78fb in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:846
#33 0x5575a25e3879 in do_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:3358
#34 0x5575a26e7eb0 in do_internal /home/mdowle/build/R-devel-strict-gcc/src/main/names.c:1385
previously allocated by thread T0 here:
#0 0x7f774e3ddbc8 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8)
#1 0x5575a26d708b in Rf_allocVector3 /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:2806
#2 0x5575a24b2656 in Rf_allocVector ../../src/include/Rinlinedfuns.h:593
#3 0x5575a24b2656 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:345
#4 0x5575a24b1ff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:206
#5 0x5575a24b1ff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:204
#6 0x5575a24b1ff9 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:338
#7 0x7f77404c0631 in copyAsPlain /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/utils.c:246
#8 0x7f774038fa5f in memrecycle /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/assign.c:721
#9 0x7f77403c52a4 in dogroups /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/dogroups.c:166
#10 0x5575a2447875 in R_doDotCall /home/mdowle/build/R-devel-strict-gcc/src/main/dotcode.c:669
#11 0x5575a259c818 in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7670
#12 0x5575a25d7787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#13 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#14 0x5575a25dce61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#15 0x5575a26edf61 in dispatchMethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:436
#16 0x5575a26ee6a5 in Rf_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:476
#17 0x5575a256eabb in tryDispatch /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:5362
#18 0x5575a258825b in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7260
#19 0x5575a25d7787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#20 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#21 0x5575a25dce61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#22 0x5575a26edf61 in dispatchMethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:436
#23 0x5575a26ee6a5 in Rf_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:476
#24 0x5575a26eec4d in do_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:565
#25 0x5575a25d7b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#26 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#27 0x5575a25dce61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#28 0x5575a25d78fb in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:846
#29 0x5575a25e3879 in do_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:3358
#30 0x5575a26e7eb0 in do_internal /home/mdowle/build/R-devel-strict-gcc/src/main/names.c:1385
#31 0x5575a25d7b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#32 0x5575a25db234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
SUMMARY: AddressSanitizer: heap-use-after-free /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:1787 in RunGenCollect
Shadow bytes around the buggy address:
0x0c2e8008f010: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e8008f020: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e8008f030: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e8008f040: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e8008f050: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c2e8008f060:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e8008f070: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e8008f080: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e8008f090: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e8008f0a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e8008f0b0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==437857==ABORTING
Thanks @venelin for the repro data and code, in data-raw.zip above. I tried that again, this time repeating 10 times. On the 8th iteration it generated the error. Great. (I ran it again to get a timing and this time it failed on the 2nd iteration in 6 mins. Great.)
$ Rdevel-strict-gcc
R Under development (unstable) (2020-07-14 r78854) -- "Unsuffered Consequences"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> library(data.table)
data.table 1.13.0 using 1 threads (see ?getDTthreads). Latest news: r-datatable.com
**********
This installation of data.table has not detected OpenMP support. It should still work but in single-threaded mode.
This is Linux. This warning should not normally occur on Windows or Linux where OpenMP is turned on by data.table's configure script by passing -fopenmp to the compiler. If you see this warning on Windows or Linux, please file a GitHub issue.
**********
> library(coda)
> load("~/Downloads/an.MCMC.1.RData")
> gctorture2(step=100)
[1] 0
> options(datatable.verbose=TRUE)
> for (i in 1:10) {
+ cat("Iteration",i,"...\n")
+
+ ans <- an.MCMC[,
+
+ list(
+
+ PostMean = mean(unlist(Mean)),
+
+ HPD = list(colMeans(do.call(rbind, HPD))),
+
+ HPD50 = list(colMeans(do.call(rbind, HPD50))),
+
+ start = start(mcs),
+
+ end = end(mcs),
+
+ thin = thin(mcs),
+
+ ESS = sum(unlist(ESS)),
+
+ G.R. = if(length(mcs)>1) {
+
+ gelman.diag(mcs, autoburnin=FALSE)$psrf[1]
+
+ } else {
as.double(NA)
},
nChains = length(mcs),
+
+ as.double(NA)
+
+ },
+
+ nChains = length(mcs),
+
+ mcmc = mcmc.list(mcmc(do.call(rbind, mcs)))),
+
+ by=list(stat, samplePriorMCMC)]
+ gc()
}+ }
Iteration 1 ...
Detected that j uses these columns: Mean,HPD,HPD50,start,mcs,end,ESS
Finding groups using forderv ... forder.c received 48 rows and 2 columns
0.224s elapsed (0.223s cpu)
Finding group sizes from the positions (can be avoided to save RAM) ... 0.063s elapsed (0.063s cpu)
lapply optimization is on, j unchanged as 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
GForce is on, left j unchanged
Old mean optimization changed j from 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) { gelman.diag(mcs, autoburnin = FALSE)$psrf[1] } else { as.double(NA) }, length(mcs), mcmc.list(mcmc(do.call(rbind, mcs))))' to 'list(.External(Cfastmean, unlist(Mean), FALSE), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
Making each group and running j (GForce FALSE) ...
memcpy contiguous groups took 14.152s for 32 groups
eval(j) took 198.144s for 32 calls
00:03:33 elapsed (00:03:33 cpu)
Iteration 2 ...
Detected that j uses these columns: Mean,HPD,HPD50,start,mcs,end,ESS
Finding groups using forderv ... forder.c received 48 rows and 2 columns
0.031s elapsed (0.031s cpu)
Finding group sizes from the positions (can be avoided to save RAM) ... 0.031s elapsed (0.031s cpu)
lapply optimization is on, j unchanged as 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
GForce is on, left j unchanged
Old mean optimization changed j from 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) { gelman.diag(mcs, autoburnin = FALSE)$psrf[1] } else { as.double(NA) }, length(mcs), mcmc.list(mcmc(do.call(rbind, mcs))))' to 'list(.External(Cfastmean, unlist(Mean), FALSE), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
Making each group and running j (GForce FALSE) ...
memcpy contiguous groups took 13.929s for 32 groups
eval(j) took 27.606s for 32 calls
42.3s elapsed (42.3s cpu)
Iteration 3 ...
Detected that j uses these columns: Mean,HPD,HPD50,start,mcs,end,ESS
Finding groups using forderv ... forder.c received 48 rows and 2 columns
0.062s elapsed (0.063s cpu)
Finding group sizes from the positions (can be avoided to save RAM) ... 0.032s elapsed (0.032s cpu)
lapply optimization is on, j unchanged as 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
GForce is on, left j unchanged
Old mean optimization changed j from 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) { gelman.diag(mcs, autoburnin = FALSE)$psrf[1] } else { as.double(NA) }, length(mcs), mcmc.list(mcmc(do.call(rbind, mcs))))' to 'list(.External(Cfastmean, unlist(Mean), FALSE), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
Making each group and running j (GForce FALSE) ...
memcpy contiguous groups took 14.009s for 32 groups
eval(j) took 28.042s for 32 calls
42.8s elapsed (42.8s cpu)
Iteration 4 ...
Detected that j uses these columns: Mean,HPD,HPD50,start,mcs,end,ESS
Finding groups using forderv ... forder.c received 48 rows and 2 columns
0.032s elapsed (0.031s cpu)
Finding group sizes from the positions (can be avoided to save RAM) ... 0.033s elapsed (0.033s cpu)
lapply optimization is on, j unchanged as 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
GForce is on, left j unchanged
Old mean optimization changed j from 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) { gelman.diag(mcs, autoburnin = FALSE)$psrf[1] } else { as.double(NA) }, length(mcs), mcmc.list(mcmc(do.call(rbind, mcs))))' to 'list(.External(Cfastmean, unlist(Mean), FALSE), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
Making each group and running j (GForce FALSE) ...
memcpy contiguous groups took 14.108s for 32 groups
eval(j) took 27.976s for 32 calls
42.9s elapsed (42.9s cpu)
Iteration 5 ...
Detected that j uses these columns: Mean,HPD,HPD50,start,mcs,end,ESS
Finding groups using forderv ... forder.c received 48 rows and 2 columns
0.065s elapsed (0.064s cpu)
Finding group sizes from the positions (can be avoided to save RAM) ... 0.032s elapsed (0.032s cpu)
lapply optimization is on, j unchanged as 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
GForce is on, left j unchanged
Old mean optimization changed j from 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) { gelman.diag(mcs, autoburnin = FALSE)$psrf[1] } else { as.double(NA) }, length(mcs), mcmc.list(mcmc(do.call(rbind, mcs))))' to 'list(.External(Cfastmean, unlist(Mean), FALSE), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
Making each group and running j (GForce FALSE) ...
memcpy contiguous groups took 14.088s for 32 groups
eval(j) took 28.184s for 32 calls
43.0s elapsed (43.0s cpu)
Iteration 6 ...
Detected that j uses these columns: Mean,HPD,HPD50,start,mcs,end,ESS
Finding groups using forderv ... forder.c received 48 rows and 2 columns
0.031s elapsed (0.032s cpu)
Finding group sizes from the positions (can be avoided to save RAM) ... 0.034s elapsed (0.034s cpu)
lapply optimization is on, j unchanged as 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
GForce is on, left j unchanged
Old mean optimization changed j from 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) { gelman.diag(mcs, autoburnin = FALSE)$psrf[1] } else { as.double(NA) }, length(mcs), mcmc.list(mcmc(do.call(rbind, mcs))))' to 'list(.External(Cfastmean, unlist(Mean), FALSE), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
Making each group and running j (GForce FALSE) ...
memcpy contiguous groups took 14.085s for 32 groups
eval(j) took 28.046s for 32 calls
42.9s elapsed (42.9s cpu)
Iteration 7 ...
Detected that j uses these columns: Mean,HPD,HPD50,start,mcs,end,ESS
Finding groups using forderv ... forder.c received 48 rows and 2 columns
0.031s elapsed (0.031s cpu)
Finding group sizes from the positions (can be avoided to save RAM) ... 0.032s elapsed (0.032s cpu)
lapply optimization is on, j unchanged as 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
GForce is on, left j unchanged
Old mean optimization changed j from 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) { gelman.diag(mcs, autoburnin = FALSE)$psrf[1] } else { as.double(NA) }, length(mcs), mcmc.list(mcmc(do.call(rbind, mcs))))' to 'list(.External(Cfastmean, unlist(Mean), FALSE), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
Making each group and running j (GForce FALSE) ...
memcpy contiguous groups took 14.624s for 32 groups
eval(j) took 28.986s for 32 calls
44.4s elapsed (44.4s cpu)
Iteration 8 ...
Detected that j uses these columns: Mean,HPD,HPD50,start,mcs,end,ESS
Finding groups using forderv ... forder.c received 48 rows and 2 columns
0.062s elapsed (0.063s cpu)
Finding group sizes from the positions (can be avoided to save RAM) ... 0.033s elapsed (0.033s cpu)
lapply optimization is on, j unchanged as 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
GForce is on, left j unchanged
Old mean optimization changed j from 'list(mean(unlist(Mean)), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) { gelman.diag(mcs, autoburnin = FALSE)$psrf[1] } else { as.double(NA) }, length(mcs), mcmc.list(mcmc(do.call(rbind, mcs))))' to 'list(.External(Cfastmean, unlist(Mean), FALSE), list(colMeans(do.call(rbind, HPD))), list(colMeans(do.call(rbind, HPD50))), start(mcs), end(mcs), thin(mcs), sum(unlist(ESS)), if (length(mcs) > 1) {'
Making each group and running j (GForce FALSE) ... =================================================================
==457456==ERROR: AddressSanitizer: heap-use-after-free on address 0x617000d75903 at pc 0x56134f7fca5f bp 0x7ffea94d6e40 sp 0x7ffea94d6e30
READ of size 1 at 0x617000d75903 thread T0
#0 0x56134f7fca5e in RunGenCollect /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:1787
#1 0x56134f7fca5e in R_gc_internal /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:3125
#2 0x56134f812edb in Rf_allocVector3 /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:2775
#3 0x56134f5eefd6 in Rf_allocVector ../../src/include/Rinlinedfuns.h:593
#4 0x56134f5eefd6 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:336
#5 0x56134f5ef0f9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:206
#6 0x56134f5ef0f9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:204
#7 0x56134f5ef0f9 in duplicate_list /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:268
#8 0x56134f5ef0f9 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:312
#9 0x56134f5efb50 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:352
#10 0x56134f5eeff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:206
#11 0x56134f5eeff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:204
#12 0x56134f5eeff9 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:338
#13 0x7f8626225631 in copyAsPlain /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/utils.c:246
#14 0x7f86260f4a5f in memrecycle /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/assign.c:721
#15 0x7f862612a2a4 in dogroups /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/dogroups.c:166
#16 0x56134f584875 in R_doDotCall /home/mdowle/build/R-devel-strict-gcc/src/main/dotcode.c:669
#17 0x56134f6d9818 in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7670
#18 0x56134f714787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#19 0x56134f718234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#20 0x56134f719e61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#21 0x56134f82af61 in dispatchMethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:436
#22 0x56134f82b6a5 in Rf_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:476
#23 0x56134f6ababb in tryDispatch /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:5362
#24 0x56134f6c525b in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7260
#25 0x56134f7232ad in R_compileAndExecute /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1514
#26 0x56134f7241b0 in do_for /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:2293
#27 0x56134f714b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#28 0x56134f7a2c35 in Rf_ReplIteration /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:264
#29 0x56134f7a3620 in R_ReplConsole /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:314
#30 0x56134f7a37b6 in run_Rmainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1113
#31 0x56134f7a3826 in Rf_mainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1120
#32 0x56134f3f7e41 in main /home/mdowle/build/R-devel-strict-gcc/src/main/Rmain.c:29
#33 0x7f862de8e0b2 in __libc_start_main (/usr/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
#34 0x56134f3fa67d in _start (/home/mdowle/build/R-devel-strict-gcc/bin/exec/R+0x7e367d)
0x617000d75903 is located 3 bytes inside of 768-byte region [0x617000d75900,0x617000d75c00)
freed by thread T0 here:
#0 0x7f862f2997cf in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
#1 0x56134f7f082e in ReleaseLargeFreeVectors /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:1114
#2 0x56134f7f082e in RunGenCollect /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:1896
#3 0x56134f7f082e in R_gc_internal /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:3125
#4 0x56134f813546 in Rf_allocVector3 /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:2626
#5 0x56134f5ef656 in Rf_allocVector ../../src/include/Rinlinedfuns.h:593
#6 0x56134f5ef656 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:345
#7 0x56134f5eeff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:206
#8 0x56134f5eeff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:204
#9 0x56134f5eeff9 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:338
#10 0x7f8626225631 in copyAsPlain /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/utils.c:246
#11 0x7f86260f4a5f in memrecycle /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/assign.c:721
#12 0x7f862612a2a4 in dogroups /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/dogroups.c:166
#13 0x56134f584875 in R_doDotCall /home/mdowle/build/R-devel-strict-gcc/src/main/dotcode.c:669
#14 0x56134f6d9818 in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7670
#15 0x56134f714787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#16 0x56134f718234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#17 0x56134f719e61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#18 0x56134f82af61 in dispatchMethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:436
#19 0x56134f82b6a5 in Rf_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:476
#20 0x56134f6ababb in tryDispatch /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:5362
#21 0x56134f6c525b in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7260
#22 0x56134f7232ad in R_compileAndExecute /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1514
#23 0x56134f7241b0 in do_for /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:2293
#24 0x56134f714b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#25 0x56134f7a2c35 in Rf_ReplIteration /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:264
#26 0x56134f7a3620 in R_ReplConsole /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:314
#27 0x56134f7a37b6 in run_Rmainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1113
#28 0x56134f7a3826 in Rf_mainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1120
#29 0x56134f3f7e41 in main /home/mdowle/build/R-devel-strict-gcc/src/main/Rmain.c:29
#30 0x7f862de8e0b2 in __libc_start_main (/usr/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
previously allocated by thread T0 here:
#0 0x7f862f299bc8 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8)
#1 0x56134f81408b in Rf_allocVector3 /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:2806
#2 0x56134f5ef656 in Rf_allocVector ../../src/include/Rinlinedfuns.h:593
#3 0x56134f5ef656 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:345
#4 0x56134f423487 in wrapper_Duplicate /home/mdowle/build/R-devel-strict-gcc/src/main/altclasses.c:1468
#5 0x56134f429291 in ALTREP_DUPLICATE /home/mdowle/build/R-devel-strict-gcc/src/main/altrep.c:228
#6 0x56134f429291 in altrep_DuplicateEX_default /home/mdowle/build/R-devel-strict-gcc/src/main/altrep.c:645
#7 0x56134f5eef52 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:283
#8 0x56134f5eeff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:206
#9 0x56134f5eeff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:204
#10 0x56134f5eeff9 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:338
#11 0x7f8626225631 in copyAsPlain /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/utils.c:246
#12 0x7f86260f4a5f in memrecycle /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/assign.c:721
#13 0x7f862612a2a4 in dogroups /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/dogroups.c:166
#14 0x56134f584875 in R_doDotCall /home/mdowle/build/R-devel-strict-gcc/src/main/dotcode.c:669
#15 0x56134f6d9818 in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7670
#16 0x56134f714787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#17 0x56134f718234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#18 0x56134f719e61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#19 0x56134f82af61 in dispatchMethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:436
#20 0x56134f82b6a5 in Rf_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:476
#21 0x56134f6ababb in tryDispatch /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:5362
#22 0x56134f6c525b in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7260
#23 0x56134f7232ad in R_compileAndExecute /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1514
#24 0x56134f7241b0 in do_for /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:2293
#25 0x56134f714b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#26 0x56134f7a2c35 in Rf_ReplIteration /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:264
#27 0x56134f7a3620 in R_ReplConsole /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:314
#28 0x56134f7a37b6 in run_Rmainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1113
#29 0x56134f7a3826 in Rf_mainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1120
#30 0x56134f3f7e41 in main /home/mdowle/build/R-devel-strict-gcc/src/main/Rmain.c:29
#31 0x7f862de8e0b2 in __libc_start_main (/usr/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
SUMMARY: AddressSanitizer: heap-use-after-free /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:1787 in RunGenCollect
Shadow bytes around the buggy address:
0x0c2e801a6ad0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e801a6ae0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e801a6af0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e801a6b00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e801a6b10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c2e801a6b20:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e801a6b30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e801a6b40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e801a6b50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e801a6b60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e801a6b70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==457456==ABORTING
Thanks @mattdowle for persisting! Really clever to put the gc() call in the loop! I wonder if the failure to reproduce in each iteration is due to a gctorture artefact or, rather, because, the error only occurs sporadically? As a further simplification, i would try to see whether it keeps reproducing if all but one line in the column list in j are commented out (just an idea).
Yes it's the start(mcs)
line. Traced it through. The mcs
column is a mcmc.list
where each item has an attribute attached. start.mcmc
from package coda
is fetching the first item of the length-3 vector stored in each item's attribute, creating a numeric column result. The mcs
list column is being copied, the whole column is copied for each and every group (no longer copies there in dev after #4655), in construction of .SD
by dogroups.c:166
before eval(j)
via copyAsPlain
on assign.c:721
. Because the list column consists of items with attributes attached, the subset of the full column copy (which was inefficient and wrong) was not sufficiently protected when assigned to the .SD column. Then if a gc occurs between .SD being created and eval(j)
getting to the start(mcs)
, which releases the copied attribute, the heap-use-after-free
occurs when start(mcs)
comes to access the free'd attribute. I don't understand why the subset of the copied column is not being protected, since SET_VECTOR_ELT is being used which should protect the item and its attribute. It seems to be related to size-1 groups as I haven't been able to reproduce when there aren't any size 1 groups. The example from POUMM has group sizes of 1's and 2's. Anyway, now that in dev that inefficient whole-column copy per group is gone, the .SD points to already-protected data in the original table without any copy, and all is well.
Have managed to make an MRE independent of POUMM, below. Difficulties overcome, for future reference:
mcmc
, but when I used 1:3
I got red herrings in the output to do with ALTREP. Hence using the non-sequence c(42,12,36)
.rnorm(90)
is the same size as the matrix in POUMM's mcmc
; i.e. dim(an.MCMC$mcs[[1]])
is [90, 1]
. Whether that's type matrix or vector doesn't matter, just its size. When I tried to make it smaller, to make a smaller example, the asan error went away and it passed fine even after 100 iterations and a gc step of 5. object.size(DT$B)/1024
is 9.5kb which I'm thinking is above a size threshold needed to tickle the heap into this state. Although only the attributes are used in the example, the whole DT$B
was being copied for each group by data.table 1.13.0, which is why the total size of the DT$B
column comes into play.gctorture(TRUE)
it passes without asan error. I thought that gctorture(TRUE)
would invoke a full gc on every single step and be the same as gctorture2(step=1)
. But that didn't seem to be the case in my testing. In fact I've found gctorture2(step=20)
to reproduce it reliably within 10 iterations, where gctorture2(step=5)
doesn't. I guess there is some path dependency going on here in the heap; e.g. not all gc's are as deep. I tried inhibit_release
but that seemed to pick up on something internal in R before I got to running this example.gctorture
to gctorture2
..(attr(B[[1]],"foo")[1])
mimics what coda:::start.mcmc.list
does, which calls coda:::start.mcmc
.So this is the test I'll add to close this one, with 5 iterations assuming that reliably reproduces the asan fault.
$ Rdevel-strict-gcc
R Under development (unstable) (2020-07-14 r78854) -- "Unsuffered Consequences"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
> library(data.table)
data.table 1.13.0 using 1 threads (see ?getDTthreads). Latest news: r-datatable.com
**********
This installation of data.table has not detected OpenMP support. It should still work but in single-threaded mode.
This is Linux. This warning should not normally occur on Windows or Linux where OpenMP is turned on by data.table's configure script by passing -fopenmp to the compiler. If you see this warning on Windows or Linux, please file a GitHub issue.
**********
> DT = data.table(A=c(1,1,2,3,3,4,5,5,6,7), B=lapply(1:10, function(x) structure(rnorm(90), foo=c(42,12,36))))
> gctorture2(step=20)
[1] 0
> ans = DT[, .(attr(B[[1]],"foo")[1]), by=A]
> ans = DT[, .(attr(B[[1]],"foo")[1]), by=A]
> ans = DT[, .(attr(B[[1]],"foo")[1]), by=A]
=================================================================
==511874==ERROR: AddressSanitizer: heap-use-after-free on address 0x61700025f483 at pc 0x558250b2aa5f bp 0x7ffd76c38980 sp 0x7ffd76c38970
READ of size 1 at 0x61700025f483 thread T0
#0 0x558250b2aa5e in RunGenCollect /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:1787
#1 0x558250b2aa5e in R_gc_internal /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:3125
#2 0x558250b4bfb5 in Rf_mkPROMISE /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:2530
#3 0x5582509eeb48 in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:6998
#4 0x558250a42787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#5 0x558250a46234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#6 0x558250a47e61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#7 0x558250a0dfc3 in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7082
#8 0x558250a42787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#9 0x558250a46234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#10 0x558250a47e61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#11 0x558250b58f61 in dispatchMethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:436
#12 0x558250b596a5 in Rf_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:476
#13 0x558250a5006f in Rf_DispatchOrEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:3614
#14 0x558250cdff9f in R_DispatchOrEvalSP /home/mdowle/build/R-devel-strict-gcc/src/main/subset.c:633
#15 0x558250cdff9f in do_subset /home/mdowle/build/R-devel-strict-gcc/src/main/subset.c:653
#16 0x558250a42b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#17 0x558250a4b98e in do_set /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:2960
#18 0x558250a42b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#19 0x558250ad0c35 in Rf_ReplIteration /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:264
#20 0x558250ad1620 in R_ReplConsole /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:314
#21 0x558250ad17b6 in run_Rmainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1113
#22 0x558250ad1826 in Rf_mainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1120
#23 0x558250725e41 in main /home/mdowle/build/R-devel-strict-gcc/src/main/Rmain.c:29
#24 0x7fe5ee8180b2 in __libc_start_main (/usr/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
#25 0x55825072867d in _start (/home/mdowle/build/R-devel-strict-gcc/bin/exec/R+0x7e367d)
0x61700025f483 is located 3 bytes inside of 768-byte region [0x61700025f480,0x61700025f780)
freed by thread T0 here:
#0 0x7fe5efc237cf in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
#1 0x558250b1e82e in ReleaseLargeFreeVectors /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:1114
#2 0x558250b1e82e in RunGenCollect /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:1896
#3 0x558250b1e82e in R_gc_internal /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:3125
#4 0x558250b4b905 in CONS_NR /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:2436
#5 0x558250a4bd24 in Rf_evalList /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:3060
#6 0x558250a42db8 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:817
#7 0x7fe5e6ab4b2e in dogroups /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/dogroups.c:183
#8 0x5582508b2875 in R_doDotCall /home/mdowle/build/R-devel-strict-gcc/src/main/dotcode.c:669
#9 0x558250a07818 in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7670
#10 0x558250a42787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#11 0x558250a46234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#12 0x558250a47e61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#13 0x558250b58f61 in dispatchMethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:436
#14 0x558250b596a5 in Rf_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:476
#15 0x558250a5006f in Rf_DispatchOrEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:3614
#16 0x558250cdff9f in R_DispatchOrEvalSP /home/mdowle/build/R-devel-strict-gcc/src/main/subset.c:633
#17 0x558250cdff9f in do_subset /home/mdowle/build/R-devel-strict-gcc/src/main/subset.c:653
#18 0x558250a42b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#19 0x558250a4b98e in do_set /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:2960
#20 0x558250a42b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#21 0x558250ad0c35 in Rf_ReplIteration /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:264
#22 0x558250ad1620 in R_ReplConsole /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:314
#23 0x558250ad17b6 in run_Rmainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1113
#24 0x558250ad1826 in Rf_mainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1120
#25 0x558250725e41 in main /home/mdowle/build/R-devel-strict-gcc/src/main/Rmain.c:29
#26 0x7fe5ee8180b2 in __libc_start_main (/usr/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
previously allocated by thread T0 here:
#0 0x7fe5efc23bc8 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8)
#1 0x558250b4208b in Rf_allocVector3 /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:2806
#2 0x55825091d656 in Rf_allocVector ../../src/include/Rinlinedfuns.h:593
#3 0x55825091d656 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:345
#4 0x558250751487 in wrapper_Duplicate /home/mdowle/build/R-devel-strict-gcc/src/main/altclasses.c:1468
#5 0x558250757291 in ALTREP_DUPLICATE /home/mdowle/build/R-devel-strict-gcc/src/main/altrep.c:228
#6 0x558250757291 in altrep_DuplicateEX_default /home/mdowle/build/R-devel-strict-gcc/src/main/altrep.c:645
#7 0x55825091cf52 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:283
#8 0x55825091cff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:206
#9 0x55825091cff9 in duplicate_child /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:204
#10 0x55825091cff9 in duplicate1 /home/mdowle/build/R-devel-strict-gcc/src/main/duplicate.c:338
#11 0x7fe5e6baf631 in copyAsPlain /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/utils.c:246
#12 0x7fe5e6a7ea5f in memrecycle /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/assign.c:721
#13 0x7fe5e6ab42a4 in dogroups /tmp/RtmpZd9HZG/R.INSTALL6acdf22c4d75a/data.table/src/dogroups.c:166
#14 0x5582508b2875 in R_doDotCall /home/mdowle/build/R-devel-strict-gcc/src/main/dotcode.c:669
#15 0x558250a07818 in bcEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:7670
#16 0x558250a42787 in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:723
#17 0x558250a46234 in R_execClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1888
#18 0x558250a47e61 in Rf_applyClosure /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:1814
#19 0x558250b58f61 in dispatchMethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:436
#20 0x558250b596a5 in Rf_usemethod /home/mdowle/build/R-devel-strict-gcc/src/main/objects.c:476
#21 0x558250a5006f in Rf_DispatchOrEval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:3614
#22 0x558250cdff9f in R_DispatchOrEvalSP /home/mdowle/build/R-devel-strict-gcc/src/main/subset.c:633
#23 0x558250cdff9f in do_subset /home/mdowle/build/R-devel-strict-gcc/src/main/subset.c:653
#24 0x558250a42b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#25 0x558250a4b98e in do_set /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:2960
#26 0x558250a42b5e in Rf_eval /home/mdowle/build/R-devel-strict-gcc/src/main/eval.c:798
#27 0x558250ad0c35 in Rf_ReplIteration /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:264
#28 0x558250ad1620 in R_ReplConsole /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:314
#29 0x558250ad17b6 in run_Rmainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1113
#30 0x558250ad1826 in Rf_mainloop /home/mdowle/build/R-devel-strict-gcc/src/main/main.c:1120
#31 0x558250725e41 in main /home/mdowle/build/R-devel-strict-gcc/src/main/Rmain.c:29
#32 0x7fe5ee8180b2 in __libc_start_main (/usr/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
SUMMARY: AddressSanitizer: heap-use-after-free /home/mdowle/build/R-devel-strict-gcc/src/main/memory.c:1787 in RunGenCollect
Shadow bytes around the buggy address:
0x0c2e80043e40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e80043e50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e80043e60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e80043e70: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e80043e80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c2e80043e90:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e80043ea0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e80043eb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e80043ec0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e80043ed0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c2e80043ee0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==511874==ABORTING
mdowle@pop-os:~/GitHub/data.table$
Thanks @mattdowle for fixing this! I would wait until 1.13.0 gets published on CRAN, and then re-submit POUMM. Is this new data.table release already planned?
Thanks for your patience, @venelin, and your help in tracking this one down. Sorry again your package was removed from CRAN because of it.
Yes, I think we're ready to release to CRAN now. There's more that would be nice to get it into this one, but we should get CRAN passing again at least. Then your Import should be data.table(>= 1.13.2)
. v1.13.0 is the version already on CRAN with the problem. (Even numbers are released to CRAN, odd numbers, currently 1.13.1, are dev.) Your package checks on CRAN might be faster now too as the list column slowdown in 1.13.0 was affecting POUMM.
Great 👌! And thanks for creating data.table!
The maintainer of package POUMM, @venelin, contacted me because this error showed up on CRAN's gcc ASAN additional check of POUMM. The root cause appears to be data.table. POUMM was removed from CRAN because it wasn't fixed within 2 weeks, which I think is unreasonable. Especially given POUMM was passing with OK and NOTE and only failing the gcc-asan additional check.
The CRAN log shows a
heap-use-after-free
originating indogroups.c
andassign.c
(all the warnings can be ignored, I think, as they're unrelated to this memory problem): https://www.stats.ox.ac.uk/pub/bdr/memtests/gcc-ASAN/POUMM/00check.logAfter some effort I've managed to reproduce a
heap-use-after-free
, albeit originating from a different part of data.table code, output below.To reproduce I needed to use gc torture together with our standard strict build of R which includes strict-barrier and ASAN. I think it is a true data.table issue at this stage.
I extracted the code from POUMM's UserGuide.Rmd and ran it with echo on and gc torture turned on just before the plot(summary(...)) thanks to input from @venelin that the data.table usage is induced by the plot.summary. Still dealing with very long run times of over 12 hours to reproduce so it's a case of whittling it down since torture is required. The
SET_VECTOR_ELT
shown on CRAN's log looks ok to me (that protects its argument being assigned) so I don't see the problem just by looking at the ASAN output, so far. Locally, the output points toduplicate.c
so I'll look at that next.