PyPy RTreeError: Error in "Index_Create": Spatial Index Error: InvalidPageException: Unknown page id 1

[mwtoews]

As discovered here, there is one test failure with PyPy. The issue can be demonstrated in a local docker container:

root@ae284360d437:/io# pytest tests/test_index.py::IndexSerialization::test_custom_filenames
==================================== test session starts ====================================
platform linux -- Python 3.10.14[pypy-7.3.16-final], pytest-8.2.2, pluggy-1.5.0
rootdir: /io
configfile: pyproject.toml
collected 1 item                                                                            

tests/test_index.py F                                                                 [100%]

========================================= FAILURES ==========================================
_________________________ IndexSerialization.test_custom_filenames __________________________

self = <tests.test_index.IndexSerialization testMethod=test_custom_filenames>

    def test_custom_filenames(self) -> None:
        """Test using custom filenames for index serialization"""
        p = index.Property()
        p.dat_extension = "data"
        p.idx_extension = "index"
        tname = tempfile.mktemp()
        idx = index.Index(tname, properties=p)
        for i, coords in enumerate(self.boxes15):
            idx.add(i, coords)

        hits = list(idx.intersection((0, 0, 60, 60)))
        self.assertTrue(len(hits), 10)
        self.assertEqual(hits, [0, 4, 16, 27, 35, 40, 47, 50, 76, 80])
        del idx

        # Check we can reopen the index and get the same results
>       idx2 = index.Index(tname, properties=p)

/io/tests/test_index.py:444: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/io/rtree/index.py:278: in __init__
    self.handle = IndexHandle(self.properties.handle)
/io/rtree/index.py:1390: in __init__
    self._ptr = self._create(*args, **kwargs)
/opt/pypy/lib/pypy3.10/_ctypes/function.py:367: in __call__
    result, forced = self._do_errcheck(result, errcheckargs)
/opt/pypy/lib/pypy3.10/_ctypes/function.py:411: in _do_errcheck
    v = self._errcheck_(result, self, args)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

result = None, func = <ctypes.CDLL.__init__.<locals>._FuncPtr object at 0x00007057b2fd0110>
cargs = (<rtree.index.PropertyHandle object at 0x00007057b2e2b050>,)

    def check_void(result, func, cargs):
        "Error checking for void* returns"
        if not bool(result):
            s = rt.Error_GetLastErrorMsg().decode()
            msg = f'Error in "{func.__name__}": {s}'
            rt.Error_Reset()
>           raise RTreeError(msg)
E           rtree.exceptions.RTreeError: Error in "Index_Create": Spatial Index Error: InvalidPageException: Unknown page id 1

/io/rtree/core.py:25: RTreeError
================================== short test summary info ==================================
FAILED tests/test_index.py::IndexSerialization::test_custom_filenames - rtree.exceptions.RTreeError: Error in "Index_Create": Spatial Index Error: InvalidPageEx...
===================================== 1 failed in 0.50s =====================================

[adamjstewart]

If PyPy is something we care about, can we test it in CI?

[mwtoews]

I don't know any PyPy users. The good news is that rtree mostly works on PyPy! Adding it to the test CI would imply to fix this issue. Furthermore, I suspect there are some difficulties with NumPy on PyPy, required by the tests, as I've seen unknown segmentation faults from pytest. And NumPyPy isn't a drop-in replacement.

I'd be interested if this bug is an Rtree implementation error that PyPy has highlighted. But I haven't looked too deep.