Crashes when using pixbuf_proc.so

[Flurrywinde]

I was looking all over for a program like this, and yours was the only one I could find. Thank you! It worked well without pixbuf_proc.so except for stuttering on large files, so I compiled pixbuf_proc.so. Now, unfortunately I get this:

$ ./infinite-image-scroller.py ~/Downloads/*.jpg --debug
2020-11-13 09:03:50 :: DEBUG :: Starting application...

(infinite-image-scroller.py:17050): Gtk-WARNING **: 09:03:50.132: Theme parsing error: gtk.css:6:26: The style property GtkRange:slider-width is deprecated and shouldn't be used anymore. It will be removed in a future version

(infinite-image-scroller.py:17050): Gtk-WARNING **: 09:03:50.132: Theme parsing error: gtk.css:7:26: The style property GtkRange:stepper-size is deprecated and shouldn't be used anymore. It will be removed in a future version
2020-11-13 09:03:50 :: DEBUG :: Adding image: /home/kanon/Downloads/2citiesright.jpg
2020-11-13 09:03:50 :: DEBUG :: Adding image: /home/kanon/Downloads/08c35-l1thesecretofbonehill.jpg
2020-11-13 09:03:50 :: DEBUG :: Adding image: /home/kanon/Downloads/9b8a694e48-scrolldown-488x57.jpg
2020-11-13 09:03:50 :: DEBUG :: pixbuf_proc [init]: /home/kanon/Downloads/2citiesright.jpg
2020-11-13 09:03:50 :: DEBUG :: pixbuf_proc [init]: /home/kanon/Downloads/08c35-l1thesecretofbonehill.jpg
2020-11-13 09:03:50 :: DEBUG :: pixbuf_proc [init]: /home/kanon/Downloads/9b8a694e48-scrolldown-488x57.jpg
2020-11-13 09:03:50 :: DEBUG :: pixbuf_proc [signal]: /home/kanon/Downloads/08c35-l1thesecretofbonehill.jpg
2020-11-13 09:03:50 :: DEBUG :: pixbuf_proc [thread]: /home/kanon/Downloads/9b8a694e48-scrolldown-488x57.jpg
2020-11-13 09:03:50 :: DEBUG :: pixbuf_proc [signal]: /home/kanon/Downloads/2citiesright.jpg
fish: Job 4, “./infinite-image-scroller.py ~/…” terminated by signal SIGSEGV (Address boundary error)

[mk-fg]

Hi, thanks.

Dunno how to reproduce the issue here without specific images or at least some idea about how they are special though - works for me on all images I've tried so far. Might also be some kind of issue with different compiler/libs and such, not just images.

SIGSEGV can mean pretty much any kind of bug in C code, unfortunately. I think you can look into it further by building the lib with -g flag, running the script with gdb, and getting a backtrace after crash, but that's a bit complicated interactive thing, and I suspect there's an easier way to get a backtrace, but I don't know of any off the top of my head, and never really looked it up.

So in abscence of additional data, don't think I can do much about it, sorry.

[mk-fg]

you can look into it further by building the lib with -g flag, running the script with gdb

Actually, one popular alternative to that (and learning gdb and such) is to instead quickly put a bunch of printf("got here 1"), printf("got here 2"), ... lines into pp_process_image_file function, then rebuild .so, and run the script with -m1 option (one bg thread, to avoid multiple outputs), and find which line it crashes on that way, then maybe printf the args to whatever call there and see what is odd about these (maybe image size somehow gets negative, something overflows, NULL somewhere, etc).

Also, if this crash happens with any images (e.g. random ones grabbed on the internets too), then guess it's likely some incompatibility with gtk libs, that might be easy to reproduce in a VM of the same distro - let me know, might be easy to try.

[Flurrywinde]

Well, I added a bunch of debugging printf's, but I couldn't find any reason for the SIGSEGV. So next I spun up a new Ubuntu 20.04.1 (My main system is Ubuntu 18.04.) in virtualbox and compiled the .so. In the vm, it works on the same image files, but if I copied that pixbuf_proc.so over to 18.04, it sigsegv's again.

Then, I moved the pixbuf_proc.so from 18.04 to 20.04, and it worked.

Thus, it seems it's not pixbuf_proc.so at all. Any idea what it could be?

[mk-fg]

Indeed it's weird. Obvious cause might just be some bug in that version/build of gtk libs, but that's unlikely, given how much they're used everywhere. Will check with gdb by booting 18.04 iso in a VM, as it should be easily reproducible that way.

[mk-fg]

Will check with gdb by booting 18.04 iso in a VM, as it should be easily reproducible that way.

Ah, nope, as that ubuntu isn't new enough for the python script even, and updating it with python3.8 ppa and to install all dev packages seem to take a while with apt and stuff like pointless locale-gen on live distro.

But given that you're probably using different python from one with which gtk3 gir bindings were generated, I think that'd likely be the problem - interface for gtk libs on 18.04 expects you to use python3.6 and its ABI, but you are using python3.8 or something like that.

If that's indeed the issue, then it's likely not worth fixing via distro packages, and maybe easier to address via chroot or container env (e.g. flatpak or appimage) with recent enough packages and no such mismatches. Dunno how to package that easily though - if there's some easy "how to AppImage stuff" guide, maybe you can follow something like that on 20.04 and then copy single self-contained AppImage binary to 18.04. That should work, I think, and what I'd probably try, if I needed the thing to work on older distro as a desktop app, but no idea if worth the effort for your use-case, of course.

[mk-fg]

using different python from one with which gtk3 gir bindings were generated, I think that'd likely be the problem

Though kinda surprising that python script works with them without the lib, and it's the .so lib (which doesn't actually use gobject-introspection) that doesn't... Weird, but still, I think it might be some lower-level pixbuff stuff that's needed to pass stuff to .so that fails in this case.

[Flurrywinde]

Yeah, I installed python 3.7 awhile back, and I seem to recall having to jump through a bunch of hoops to get it working right. Thus, when infinite-image-scroller didn't work under python 3.6, I switched over to 3.7.

I'll see if I can make an appimage.

[mk-fg]

If you'll find an easy guide for how to make it or can just list steps, maybe link it here too.

I can probably re-run them in a VM and attach result under releases here somewhere, so that someone else trying to build the thing on ubuntu won't need to install 1000 dev packages (build-essential + whole libgtk-3-dev + python-dev) just to build one tiny .so lib, and can use such AppImage instead. (not sure if flatpak might be easier, but afaik unlike AppImage binaries it involves flathub and more administrivia to publish)

[Flurrywinde]

Ok back from a bunch of research on making an appimage. It seems doable but with a significant learning curve. When I accomplish it, I'll let you know, but in the meantime, I got things working under Ubuntu 18.04 by figuring out how to get gi and cairo working in python 3.8. So now pixbuf_proc.so works! But... the stuttering issue remains even after I increased threads beyond the number of cpus I have. When the next image is loaded, the images jump ahead sometimes (this is when using -a 1:.01 as a command line parameter). Any idea why or what can be done about it?

Let me know if you'd like more details of my appimage research or how to get multiple versions of python working. All this is a bit over my head still, but I'm liking the learning process.

[mk-fg]

But... the stuttering issue remains even after I increased threads beyond the number of cpus I have.

Maybe need to increase queue length for loading images, e.g. "-q20" or something like that. Not sure what it might be blocking on otherwise - just displaying pre-resized images shouldn't block gtk on any reasonably modern hardware.

Jumping might also be due to queue, I guess, if things get added/removed near the current point. But also might be due to something different in gtk theme I guess, as given how this is implemented by controlling scrollbar, some kind of added margins or such constraints might screw things up a bit, I imagine. I think less flaky way to implement such scrolling would be to forego all GUI toolkits and just render the scroll via SDL or something, like a game would, but that'd be an entirely different project.

Dunno about other causes for stuttering - if .so module is loaded and works, only thing happening in the main thread should be adding pre-scaled gtk pixbufs to a window, and if it can't handle that without blocking - oh well, probably can't be helped.

Also, wrt thread count - increasing number of these past cpu threads might affect performance of the main one, so probably not a good idea, even if loading images can't be done fast enough for a given scroll speed with cpu power available. (maybe switch to use a different scaling algo or pre-scale them first via "convert" - that gotta be the only expensive thing there)

[mk-fg]

maybe switch to use a different scaling algo

Default for -z/--scaling-interp seem to be billinear already, which I think should be fastest one as it is.

[mk-fg]

But... the stuttering issue remains even after I increased threads beyond the number of cpus I have.

Oh, also, probably obvious, but I'd suspect that it's just .so file not being loaded.

Running it as ./infinite-image-scroller.py -rl --pos=M1 -x=-decorated -s0 -q20 -a 1:0.001 images (10x faster) for a random 5G dir of wallpapers, can't seem to notice any kind of issues on my mid-range desktop machine from ~2011, so unless you're running the thing on a much larger images (or maybe 4k+ display - scaling up should also be non-free without gpu help) or much weaker hw (like arm boards), don't think performance bottleneck should be an issue.