Reflections on PySide vs. Kivy

[mihow]

Hello, thanks for your great work in this application. I am currently building an desktop app related to processing insect photographs from traps and I came across your project. I have been using Kivy but I see you have recently refactored naturtag to use PySide instead. I'm curious how development has been going with PySide and if you are happy with the decision. I have more experience with web applications and I see that PySide can use stylesheets for layouts, which is appealing. I also see you have tried a few approaches to creating asynchronous background tasks. I've been experimenting with approaches for this as well and have been surprised how under-documented async actions are. It seems like a design pattern that would be common for making responsive feeling apps. Thanks for any thoughts you care to share. I hope to make some contributions to naturtag myself.

[JWCook]

Those are some good questions. It has been kind of interesting to learn two different GUI frameworks and see the different approaches they take.

Overally I liked Kivy, and I'd still consider using it again for a simpler app. I'd say it was about 70% pleasant to work with and 30% painful. The parts I liked were the the more pythonic API (compared to Qt), and the kv language that makes it convenient to keep the view separated from the application logic.

One of the parts I didn't like was debugging; I probably spent about a third of my development time trying to figure out things that weren't well covered in the docs, and fighting with obtuse and uninformative tracebacks (although I think that's been improved in 2.0+).

Regarding the learning curve, I think the docs and examples do a great job up to maybe a beginner to intermediate level, but I felt like there was a sudden scarcity of information when trying to venture beyond that and do anything moderately complex (including features I wanted to make for this app).

The default widgets and theme look fairly dated. I was using KivyMD, which has much prettier and more modern-looking widgets, but I got a bit frustrated with all the rough edges in its API. It seems promising but far from production-ready, and hasn't progressed much in the 2 years since I first used it.

I think performance was what finally made me give up on Kivy, at least for this app. Even with async and multithreaded processing, and reserving the main thread for mostly just the GUI, I just couldn't get it to be responsive and performant enough for the quantity of data and widgets I wanted to work with. It's very likely I didn't use it to its full potential and there were more performance gains to be made, but if so they still eluded me after months of trying.

[JWCook]

After taking a long break from it, I decided to try out Qt via PySide6. I'm glad I did, but first impressions weren't great. The API looks more like the underlying C++ interface than idomatic python, and does some weird things like heavy use of overloading instead of keyword args. My experience with the docs has been similar to Kivy so far, in that it's good up the beginner/intermediate level, but beyond that there is a serious lack of usable examples. Early on I had to spend a lot of time translating C++ examples to python. A bit more discussion on that here. I picked up this book, which has helped a lot in areas where the docs are lacking, but even that doesn't cover everything.

The overall sophistication of features seems uneven; in some places it has surprisingly useful widgets that "just work," and do everything you want out of the box, like menus, toolbars, and native file choosers that are much more fully-featured and polished than Kivy's equivalents. But then for other seemingly simple tasks, it feels like you're handed half a box of Ikea furniture parts, a 3D printer, and a soldering iron. I was especially frustrated with the barebones selection of layout classes. Kivy does more out of the box in that department. Debugging is sometimes difficult, since different objects may be owned by either python or Qt in different scenarios, and when an error happens in Qt-land (usually memory access-related), you get a segfault instead of a friendly traceback.

After using it for a few months, though, I'm actually liking it quite a bit. The API may be a bit more difficult to learn, but it's fairly consistent, so figuring out advanced usage of one widget often involves concepts that apply to several others. I miss Kivy's kvlang, but QSS is nice. The signals and slots system is good, comparable to Kivy's event dispatch system, but more powerful. So far my event communication across various parts of the app is a lot less spaghettified than my Kivy version was, although that's partly just from learning from mistakes I made the first time around.

Finally, the performance is much better, at least for my use case. I'm still struggling a bit with concurrency in a few areas, but the multithreading interface is good. The main limitation I've recently run into is that there is no multiprocessing equivalent. There's this class, but it's meant for communicating with an external application (more like python's subprocess module), not parallel processing for a large batch of CPU-bound tasks.

I am currently building an desktop app related to processing insect photographs from traps and I came across your project.

That sounds cool, can you tell me more about it?

[JWCook]

@mihow Does that answer your question?

[mihow]

Hi @JWCook, I received a notification about this ticket when you closed it. I am truly sorry for never responding after your very thoughtful and fast reply. Looking at my calendar, I got a bad case of C-19 right after I asked this question. So I will blame my silence and forgetfulness on the delirium I endured. I look forward to trying the latest version of naturtag. I did end up using Kivy for our insect ML project, but with frustration, especially in regards to the UI layouts. I haven't found a good logical way to re-use database connections and run long-running background tasks from the app. I also intertwined the view and application logic too much. I have since moved to a web based UI with a decoupled backend. It can only run in the browser as of now, but I am curious about using pywebview to embed it in a desktop app. Here is our current Kivy app: https://github.com/RolnickLab/ami-data-companion

[JWCook]

Hey, no problem, and it's good to hear from you.

I haven't found a good logical way to re-use database connections and run long-running background tasks from the app

One option would be to store your SQLAlchemy session (or equivalent db session/connection) object on your main App class and access that from your views via get_running_app().

Switching to a web framework is a totally sensible option, though. I considered that for awhile too, but I think I'm pretty determined to make a native desktop app.

Pywebview looks neat. Is that kind of in the same category as Electron? And it looks like your app is coming along nicely! Just curious, is this project intended for ecology/biodiversity research, or agronomy, or something else?

[JWCook]

Oh, and the next release of naturtag is probably still a few months away. It's been kinda stuck at (what feels like) 90% done for awhile, mostly just due to being busy on other projects and also feature creep. You know how it goes. It's still (slowly) coming along, though!

[mihow]

Hey, no problem, and it's good to hear from you.

Likewise!

I haven't found a good logical way to re-use database connections and run long-running background tasks from the app

One option would be to store your SQLAlchemy session (or equivalent db session/connection) object on your main App class and access that from your views via get_running_app().

Thanks, I think I will try this very soon. My main workstation isn't fazed by thousands of new db connections and I didn't notice this issue for a long time. But when my colleagues use the app, it gets choked up on large numbers of images.

Switching to a web framework is a totally sensible option, though. I considered that for awhile too, but I think I'm pretty determined to make a native desktop app.

I do like the advantages of a native app, especially when working with multimedia.

Pywebview looks neat. Is that kind of in the same category as Electron?

Yes, maybe simpler than Electron, but for Python!

And it looks like your app is coming along nicely! Just curious, is this project intended for ecology/biodiversity research, or agronomy, or something else?

Mainly focused on biodiversity monitoring. But we certainly see applications in agronomy. One intent of the web/online version is to share data across deployments and allow for things like early warning detections of invasive species. Eventually!

[JWCook]

Oh, that sounds really cool! And definitely an area I'm interested in.

At my day job, some coworkers have worked with tools with similar goals, but proprietary (for example FarmSense), and focused on a small subset of agricultural pests. Also, I briefly worked on a hackathon project to use public observation data (from iNat and GBIF) to supplement traditional forecast methods to try to more accurately predict the emergence of Fall Armyworm in a given location (it only got as far as a half-working demo, though!). Oh, and pyinaturalist originally started as part of a project to help monitor the spread of invasive Asian hornets in Belgium (not by me; the original author was @niconoe working with INBO).

Anyway, I just work in software/data engineering and don't have any entomology background myself, but monitoring pests/invasives is an area I've been on the periphery of. It's cool to see more open-source projects in that space!