Calling XLENGTH on S4 objects

[fikovnik]

Running the following code (apart from #1288):

A <- setRefClass("A")
A$methods(f = function(i) { print(i) })

bug <- TRUE

# class B inherits from A
B <- setRefClass("B", contains = "A")
B$methods(
  # destructor that sets the global flag `bug` to FALSE
  finalize = function() { bug <<- FALSE },

  # method that calls `f` from the superclass
  g = function() {
    usingMethods("f")
    lapply(1, "f")
})

# new instance of class B
b <- B()

# class B$g which calls A$f priting 1
b$g()

# removes b
rm(b)

# since b is not referenced from anywhere, it should be collected
# and its finalized called
gc()

# check that the finalized has run
stopifnot(!bug)

also triggers:

$ PIR_ENABLE=off bin/R -d valgrind -f gcbug.R
...

==3026514== Conditional jump or move depends on uninitialised value(s)
==3026514==    at 0x858CAA1: record (TypeFeedback.h:192)
==3026514==    by 0x858CAA1: record_type (TypeFeedback.h:360)
==3026514==    by 0x858CAA1: rir::evalRirCode(rir::Code*, SEXPREC*, rir::CallContext const*, rir::Opcode*, rir::BindingCache*) (interp.cpp:2340)
==3026514==    by 0x25037B: Rf_eval (eval.c:762)
==3026514==    by 0x25218D: R_execClosure (eval.c:1958)
==3026514==    by 0x2532CB: R_execMethod (eval.c:2137)
==3026514==    by 0x48835A8: R_dispatchGeneric (methods_list_dispatch.c:1145)
==3026514==    by 0x296812: do_standardGeneric (objects.c:1290)
==3026514==    by 0x85968B4: rir::doCall(rir::CallContext&, bool) (interp.cpp:946)
==3026514==    by 0x858CA22: rir::evalRirCode(rir::Code*, SEXPREC*, rir::CallContext const*, rir::Opcode*, rir::BindingCache*) (interp.cpp:2361)
==3026514==    by 0x8596167: rir::rirCallTrampoline(rir::CallContext const&, rir::Function*, SEXPREC*, SEXPREC*) (interp.cpp:504)
==3026514==    by 0x8596FD8: rir::doCall(rir::CallContext&, bool) (interp.cpp:1126)
==3026514==    by 0x858D879: rir::evalRirCode(rir::Code*, SEXPREC*, rir::CallContext const*, rir::Opcode*, rir::BindingCache*) (interp.cpp:2448)
==3026514==    by 0x8596167: rir::rirCallTrampoline(rir::CallContext const&, rir::Function*, SEXPREC*, SEXPREC*) (interp.cpp:504)

...

==3026514== Conditional jump or move depends on uninitialised value(s)
==3026514==    at 0x858A661: rir::evalRirCode(rir::Code*, SEXPREC*, rir::CallContext const*, rir::Opcode*, rir::BindingCache*) (interp.cpp:3593)
==3026514==    by 0x8596167: rir::rirCallTrampoline(rir::CallContext const&, rir::Function*, SEXPREC*, SEXPREC*) (interp.cpp:504)
==3026514==    by 0x8596FD8: rir::doCall(rir::CallContext&, bool) (interp.cpp:1126)
==3026514==    by 0x858CA22: rir::evalRirCode(rir::Code*, SEXPREC*, rir::CallContext const*, rir::Opcode*, rir::BindingCache*) (interp.cpp:2361)
==3026514==    by 0x25037B: Rf_eval (eval.c:762)
==3026514==    by 0x25218D: R_execClosure (eval.c:1958)
==3026514==    by 0x252FA4: Rf_applyClosure (eval.c:1884)
==3026514==    by 0x23EE60: bcEval (eval.c:7152)
==3026514==    by 0x2503AF: Rf_eval (eval.c:759)
==3026514==    by 0x25218D: R_execClosure (eval.c:1958)
==3026514==    by 0x252FA4: Rf_applyClosure (eval.c:1884)
==3026514==    by 0x23EE60: bcEval (eval.c:7152)

The problem is that we call XLENGTH on S4SXP which in release mode (USE_RINTERNALS) looks at uninitialized data (right after the S4SXP). Debug mode uses the XLENGTH from memory.c which calls CHK2 that properly reports the error.