Virtual ListCtrl does not reliably render more than about 364k rows

[CypherGrue]

Operating system: Ubuntu 23.10 wxPython version & source: 4.2.1 (distro python3-wxgtk4.0 4.2.1+dfsg-1) Python version & source: 3.11.6 (distro python3 3.11.4-5)

Description of the problem:

Virtual ListCtrl does not reliably render more than about 364k rows

import wx

class ListCtrl(wx.ListCtrl):
    def __init__(self, parent, ident, style):
        wx.ListCtrl.__init__(self, parent, ident, style=style)
        self.InsertColumn(0, 'Column', width=550)

    def OnGetItemText(self, i, col):
        return str(int(i))

class Frame(wx.Frame):

    def __init__(self, *args, **kwargs):
        super(Frame, self).__init__(*args, **kwargs)
        self.coinlist = ListCtrl(self, wx.ID_ANY, wx.LC_REPORT|wx.LC_VIRTUAL)
        self.coinlist.SetItemCount(400*1000)
        self.Show()

if __name__ == '__main__':
    app = wx.App()
    Frame(None, -1, 'Window', size=(600,300))
    app.MainLoop()

[reticulatus]

I tested your code using Python 3.10.12 + wxPython 4.2.1 gtk3 (phoenix) wxWidgets 3.2.2.1 on Linux Mint 21.3 and it worked fine, even when I increased the item count to 1000*1000.

The main differences between our setups is that I am using Python 3.10 and you are using 3.11, plus I installed wxPython 4.2.1 using pip, whereas you are using the version from your distribution.

[CypherGrue]

>>> wx.__version__
'4.2.1'
>>> wx.wxWidgets_version
'wxWidgets 3.2.2'

Probably not a python issue. Same in C++:

#include <wx/wx.h>
#include <wx/listctrl.h>

class ListCtrl : public wxListCtrl
{
public:
    ListCtrl(wxWindow *parent, wxWindowID id, long style)
        : wxListCtrl(parent, id, wxDefaultPosition, wxDefaultSize, style)
    {
        InsertColumn(0, "Column", wxLIST_FORMAT_LEFT, 550);
    }

    wxString OnGetItemText(long item, long col) const
    {
        return wxString::Format(wxT("%ld"), item);
    }
};

class Frame : public wxFrame
{
public:
    Frame(const wxString &title, const wxSize &size)
        : wxFrame(NULL, wxID_ANY, title, wxDefaultPosition, size)
    {
        coinlist = new ListCtrl(this, wxID_ANY, wxLC_REPORT | wxLC_VIRTUAL);
        coinlist->SetItemCount(400 * 1000);
        Show(true);
    }

private:
    ListCtrl *coinlist;
};

class MyApp : public wxApp
{
public:
    virtual bool OnInit()
    {
        Frame *frame = new Frame("Window", wxSize(600, 300));
        return true;
    }
};

wxIMPLEMENT_APP(MyApp);

Although my system appears to have a broken wx-config:

g++ -o wxtest wxtest.cpp `wx-config --cxxflags --libs` 
/usr/bin/ld: cannot find -lwx_gtk3u_xrc-3.0: No such file or directory
/usr/bin/ld: cannot find -lwx_gtk3u_html-3.0: No such file or directory
/usr/bin/ld: cannot find -lwx_gtk3u_qa-3.0: No such file or directory
/usr/bin/ld: cannot find -lwx_gtk3u_adv-3.0: No such file or directory
/usr/bin/ld: cannot find -lwx_gtk3u_core-3.0: No such file or directory
collect2: error: ld returned 1 exit status

So the way I build is:

g++ -o wxtest wxtest.cpp -I/usr/lib/x86_64-linux-gnu/wx/include/gtk3-unicode-3.2/ -I/usr/lib/x86_64-linux-gnu/wx/include/gtk3-unicode-3.2/wx -I/usr/include/wx-3.2 -L/usr/lib/x86_64-linux-gnu/ -D_FILE_OFFSET_BITS=64 -DWXUSINGDLL -D__WXGTK__ -pthread -lwx_gtk3u_xrc-3.2 -lwx_gtk3u_html-3.2 -lwx_gtk3u_qa-3.2 -lwx_gtk3u_adv-3.2 -lwx_gtk3u_core-3.2 -lwx_baseu_xml-3.2 -lwx_baseu_net-3.2 -lwx_baseu-3.2

[reticulatus]

What version of GTK3 do you have?

richardt@Pavilion:~ $ dpkg -l  libgtk-3-0
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name             Version          Architecture Description
+++-================-================-============-====================================
ii  libgtk-3-0:amd64 3.24.33-1ubuntu2 amd64        GTK graphical user interface library

[CypherGrue]

$ dpkg -l  libgtk-3-0
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name             Version          Architecture Description
+++-================-================-============-====================================
ii  libgtk-3-0:amd64 3.24.38-5ubuntu1 amd64        GTK graphical user interface library
ii  libgtk-3-0:i386  3.24.38-5ubuntu1 i386         GTK graphical user interface library

[CypherGrue]

Tried downgrading libgtk-3-0 to 3.24.33-1ubuntu2, libgtk-3-0-dev to 3.24.33-1ubuntu1, then rebuilt and ran, issue still present.

Have tried gnome, cinnamon desktop envs with xorg and not.

Have noticed that just at the position where rows begin disappearing, if I shrink the window vertically it causes the rows to reappear. But this does not work further down. If I resize the window back up, they disappear again.

[CypherGrue]

Tested that my overlay scrollbar setting is not causing this.

I took ubuntu VM images from linuxvmimages.com (not affiliated nor recommended), then saved wxtest.cpp to the Desktop and ran:

sudo apt install libwxgtk3.2-dev
g++ -o wxtest wxtest.cpp `wx-config --cxxflags --libs`  && ./wxtest

On 23.10 (libgtk-3-0 3.24.38-5ubuntu1, libwxgtk3.2-dev 3.2.2+dfsg-4), issue is present

On 23.04 (libgtk-3-0 3.24.37-1ubuntu1, libwxgtk3.2-dev 3.2.2+dfsg-2build1), issue is absent

[CypherGrue]

Downgrading every package in the dependency tree, the culprit appears to be libcairo2

libcairo2_1.18.0-1_amd64.deb: issue is present libcairo2_1.16.0-7_amd64.deb: issue is absent

[CypherGrue]

cairo 1.17.4 good cairo 1.17.6 good cairo 1.17.8 bad cairo 1.18.0 bad

[CypherGrue]

cairo 5dafd741 good cairo 9fbf8f48 good cairo 47a21c6e bad FAULT Clamp path coordinates cairo 451dcd31 bad cairo ccb306b8 bad cairo 91364546 bad cairo ff08edfc bad cairo 4913f079 bad cairo aeafbf55 bad

[CypherGrue]

I don't know much about guis, but it looks like cairo change 451dcd31 clamps double values to fixed point values, but the 32-bit fixed point type uses 8 bits for the fractional part, leaving 24 bits for pixels. Dividing the pixels by the number of rows in my list yields 2^24/364000=46.09 pixels per row. And I have 46/2=23 pixels per row on my screen, I guess half the range being used for negative numbers. Observation seems to confirm the bisection, my list is being clamped by this change.

[swt2c]

Really nice work @CypherGrue in troubleshooting the issue. Unfortunately, I don't think we can do much to help in wxPython, but I'd suggest bringing your issue upstream to the cairo devs.

[CypherGrue]

Thanks.

Raised https://gitlab.freedesktop.org/cairo/cairo/-/issues/824

[CypherGrue]

This boomerang back down. Cairo dev suggests that the issue is that the library is not being used correctly. https://gitlab.freedesktop.org/cairo/cairo/-/issues/824#note_2264420

Maybe the wx virtual window is too big or something, or an issue in gtk? I am going to drop this here, as no more time. Good luck.