Creating an std.vector from a slice accesses wrong memory

[charlieb]

It appears that if I create a c++ vector iterator from a python slice the memory management is a little wonky. If I don't slice and just use the python list it works great. If I use the python list and increment the iterator it also works great.

#include <ios>
#include <iostream>
#include <vector>

void test(std::vector<std::uint8_t>::iterator& it) {
  for (int i = 0; i < 10; i++) {
    printf("0x%02X, ", *it++);
  }
  printf("\n");
}

import cppyy
from cppyy.gbl import std
import cppyy.ll

def main():
    cppyy.load_library('libtest.so')
    cppyy.include('../test.cpp')

    global uint8_t
    uint8_t = cppyy.gbl.uint8_t

    v = std.vector[uint8_t](list(range(20)))
    cppyy.gbl.test(v.begin())
    cppyy.gbl.test(v[2:].begin())
    cppyy.gbl.test(v.begin() + 2)

if __name__ == '__main__':
    main()

Output:

$ python ../test.py 
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 

note the middle line, that's the vectorized slice. The other two are correct.

[wlav]

Is a reference counting problem. After v[2:].begin(), the temporary produced by v[2:] gets ref-counting to zero, deleting its memory, while the iterator returned from begin() still points into it. Python ref-counting is a bit more aggressive than C++ temporaries, as the latter are kept alive until the end of a statement, the former have no such guarantee.

cppyy sets automatic lifelines when methods return members of a temporary object, but it doesn't detect returns of pointers into that temporary. It should be easy enough to provided that for methods such as std::vector<>::begin().

Note that the slicing creates a new std::vector<>, so v.begin()+2, which indexes the original vector, is way cheaper for larger vectors.