Explosion at the sparseList

[kondziu]

I ran the following code:

library(microbenchmark)
library(ggplot2)

library(ufovectors)
library(ufoaltrep)

setwd("~/Workspace/ufo_workspace/UFOs/ufovectors/benchmark")

result1 <- microbenchmark(
  "UFO" = {
    sum(ufo_integer_bin("32Mints.bin"))
  },
  "ALTREP" = {
    sum(altrep_ufo_integer_bin("32Mints.bin"))
  },
  times = 10L
)

This caused the following explosion:

R: sparseList.c:344: listAdd: Assertion `actualIdx+1 == l->usedSlots || 1 == eCompare(l->list[actualIdx+1].ptrI, e)' failed.

Thread 9 "R" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffdec7f700 (LWP 8826)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff589b801 in __GI_abort () at abort.c:79
#2  0x00007ffff588b39a in __assert_fail_base (fmt=0x7ffff5a127d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", 
    assertion=assertion@entry=0x7fffeaa81850 "actualIdx+1 == l->usedSlots || 1 == eCompare(l->list[actualIdx+1].ptrI, e)", file=file@entry=0x7fffeaa81670 "sparseList.c", 
    line=line@entry=344, function=function@entry=0x7fffeaa818d0 <__PRETTY_FUNCTION__.2739> "listAdd") at assert.c:92
#3  0x00007ffff588b412 in __GI___assert_fail (assertion=0x7fffeaa81850 "actualIdx+1 == l->usedSlots || 1 == eCompare(l->list[actualIdx+1].ptrI, e)", file=0x7fffeaa81670 "sparseList.c", 
    line=344, function=0x7fffeaa818d0 <__PRETTY_FUNCTION__.2739> "listAdd") at assert.c:101
#4  0x00007fffeaa809ea in listAdd (list_t=0x555556c61df0, ptr=0x7fffb11c7000, length=128004096, value=0x5555569aa980) at sparseList.c:344
#5  0x00007fffeaa7eb61 in allocateUfo (i=0x555556ce5230, msg=0x7fffdec7ede0) at userfaultCore.c:248
#6  0x00007fffeaa7eeee in readHandleMsg (i=0x555556ce5230) at userfaultCore.c:317
#7  0x00007fffeaa7f0e7 in handler (arg=0x555556ce5230) at userfaultCore.c:371
#8  0x00007ffff5c536db in start_thread (arg=0x7fffdec7f700) at pthread_create.c:463
#9  0x00007ffff597c88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

[kondziu]

Further investigation shows that I can (always?) get the same thing to happen if I execute this:

library(ufovectors)

setwd("~/Workspace/ufo_workspace/UFOs/ufovectors/benchmark")
ufo_set_debug_mode(T)

m <- ufo_integer_bin("32Mints.bin")
rm(m)                                               ## removes the reference
gc()                                                   ## runs the garbage collector
ufo_integer_bin("32Mints.bin") ## explodes

[kondziu]

info about l from the moment of crash:

(gdb) p $2->allocatedSlots 
$5 = 8
(gdb) p $2->size 
$6 = 1
(gdb) p $2->usedSlots 
$7 = 4
(gdb) p $2->modCt
$8 = 11
(gdb) p $2->list 
$9 = (entryI *) 0x55555742bdf0
(gdb) p $2->list->length 
$10 = 128004096
(gdb) p $2->list->occupied 
$11 = 1
(gdb) p $2->list->ptr
$12 = (void *) 0x7fffe83ed000
(gdb) p $2->list->ptrI
$13 = 140737089818624
(gdb) p $2->list->valuePtr 
$14 = (void *) 0x555556404ed0

[kondziu]

I ran it again and collected information relevant to the assert:

actualIdx                 = 0
l->usedSlots              = 9
l->list[actualIdx+1].ptrI = 140736122564608
e                         = 140737087692800
eCompare == -1            = 1
eCompare == 1             = 0

[kondziu]

It looks like whenever it explodes, before it explodes, when the gc is destroying the UFO, the UFO has the wrong file path:

__destroy
    source file: expand.dots
    vector type: 13
    vector size: 32000000
   element size: 4

it should be this:

    source file: 32Mints.bin
    vector type: 13
    vector size: 32000000
   element size: 4

This might mean that the error lies somewhere outside of sparseList, and rather is something related to a criminal use of strings on my part.

[kondziu]

I fixed the path problem, but sparseList still explodes.

[kondziu]

After https://github.com/PRL-PRG/UFOs/pull/13 same scenario explodes as follows:

R: sparseList.c:346: listAdd: Assertion `1 == eCompare(l->list[actualIdx+1].occupied, l->list[actualIdx+1].ptrI, e)' failed.