Tangent128 / luasdl2

A pure C binding of SDL 2.0 for Lua 5.1, Lua 5.2, and LuaJIT.
ISC License
383 stars 73 forks source link

SDL HitTests will make window invalid. #75

Closed tDwtp closed 3 years ago

tDwtp commented 5 years ago

I tried to use HitTests (in lua 5.2 with SDL 2.0.7). After some mouse movement inside the window, it suddenly turns invalid.

I made a minimal example here:

local sdl = require "sdl"

do -- initialization
    local ok, msg = sdl.init {sdl.flags.Everything}
    if not ok then
        msg = "sdl: " .. tostring (msg)
        error (msg)
    end
end

-- window creation
local window = sdl.createWindow {
    title  = "HitTest Bug",
    width  = 240,
    height = 240,
    flags  = {
        sdl.window.Resizable
    }
}

do -- hittest
    local Draggable   = sdl.hitTestResult.Draggable

    local function hittest (window, area)
        print (window)
        print (string.format ("area: %dx%d", area.x, area.y))
        return Draggable
    end

    window:setHitTest (hittest)
end

-- main loop
local running = true
while running do
    -- event "sdl.pollEvent"
    for event in sdl.pollEvent ( ) do
        if event.type == sdl.event.Quit then
            running = false
        end
    end

    -- error display
    local err = sdl.getError ( )
    if err and #err > 0 then
        print(err)
        sdl.clearError ( )
        running = false
    end
end

expected output:

[...]
window "HitTest Bug": size 240x240
area: 231x204
window "HitTest Bug": size 240x240
area: 230x204
window "HitTest Bug": size 240x240
area: 229x204
window "HitTest Bug": size 240x240
area: 229x203
Invalid window
tDwtp commented 5 years ago

I think I found the error: It occurs as soon as the garbagecollector runs. edit 1: adding collectgarbage("collect") directly after the polling loop makes it even more visible.

I assume it is due to the flag mustdelete which is set to 1 by commonPushUserdata. However that window is still in use!

I am not quite confident about my assumption here. I guess mustdelete is used to determin, whether the underlying object must be freed, no the CommonUserdata per se.

edit 1: I recompiled with my fix, but it did not work? edit 2: Changed the fix; its working now. If I did not undersand the mustdelete-flag correctly, it may cause memory leaks!