[kivy + pyd] Debugging InvalidMemoryOperationError

[tmarplatt]

Hello. First and foremost congrats on a terrific job with pyd and I hope it continues to receive support in the future. Having delved in D code in the past I wanted to give pyd a shot while also trying to use kivy as a python frontend, with the goal of learning to integrate D and python by writing a simple game.

I've run into InvalidMemoryOperationError errors several times when using kivy with pyd so far. At first I wanted to test how performant it was to assign values to a D array thousands times per second and converting it to a Python array before sending to pyd, versus assigning directly from D on a Python array that is a member of a Kivy widget. I found out that the latter lead invariably to these errors, while the former was usually succesful. I thought I had worked around this by properly making calls to pyd() to assign values, but I've ran into the same problem when writing my game "engine".

The .d script is as simple as this:

module ZombieWorldEngine;

import pyd.pyd;

import std.random;
import std.algorithm : canFind;
import std.conv : to;

import std.stdio : writeln;

enum maxVelocityVector = 10.0f;

void ZombieWorldEngine(PydObject _HCC, PydObject components, PydObject changed, PydObject inputQueue) {
    // Post-input: compare queue to inputSignals enum, then act on the PlayerUnit widget
    auto moveSignals = [ 
        _HCC.inputSignals.moveLeft, 
        _HCC.inputSignals.moveRight, 
        _HCC.inputSignals.moveUp, 
        _HCC.inputSignals.moveDown,
        _HCC.inputSignals.runLeft,
        _HCC.inputSignals.runRight,
        _HCC.inputSignals.runUp,
        _HCC.inputSignals.runDown
    ];
    // Find PlayerUnit widget component
    PydObject playerComp;
    foreach (comp; components) {
        if (comp.state.has_key("life")) {
            playerComp = comp;
            break;
        }
    }
}

// Expose functions to python
extern(C) void PydMain() {
    def!(ZombieWorldEngine)();

    module_init();
}

All kivy does with it is run ZombieWorldEngine() 60 times per second, while passing some Python objects. None of them are kivy objects, but components and changed are member objects of kivy Widgets.

I edited setup.py to compile the script with -g, set up gdb and set a breakpoint on this recurring error. I attached the gdb log output.

It seems to be failing in libphobos2.so.0.74. I should mention I'm currently using dmd v2.074.0.

gdb.txt

Edit: I forgot to mention what kivy is doing with ZombieWorldEngine().

[tmarplatt]

I've been reading up on D's Garbage collector. It seems that the InvalidMemoryOperationError happens when calling the Dtor of PydObject, which triggers a call to Py_DECREF(). There even is a comment in the source of deimos/python/object.d that warns against the current implementation of PyDECREF().

In my case I've reduced the problem down to the foreach statement. Just iterating over the components PydObject eventually leads to an InvalidMemoryOperationError. components is simply a Python list of kivy Widget objects.

[ariovistus]

do you have this as a full project + build somewhere? That would probably be the easiest way for me to figure out what's going on

[tmarplatt]

I'm sorry, I don't. But I wrote this test case, which strangely enough doesn't crash with InvalidMemoryOperation but with a segfault (_assert error "refcount negative" in PyDECREF()), and which might interest you still. It only needs kivy: pip install kivy. I'm using Python 3.4.3 and D v2.079.0.

https://gist.github.com/tmarplatt/29c7025e42784c2c539d45bfcfaac5c7

I found out replacing the foreach statement with for fixes any errors, like: for (uint i = 0; i < a.length(); i++) { current = testPO(a[i]); }

I'll try and produce an InvalidMemoryOperation error test case soon.