Discussion: App sometimes hangs, slow rendering, thoughts? (Also: what Qt could learn from React/Redux/Relay)

[LouisStAmour]

I've noticed that (a) the app takes a long time to load (with a white screen), then after 3 or so seconds it displays, (b) when running in slower Debug mode, the app visibly pauses without accepting user input in places where things might not be as optimized (where code runs on the main thread slower than it normally would?) and (c) when animations are disabled, and you use keyboard shortcuts for next/previous, you can finish two commands before the screen changes once, let alone twice. This is more visible (slower) going forward than going backward.

I'm used to doing React front-end work, Chrome event loop optimization and all that, so I was confused and amazed both that Qt in 2020 is still written to interactively paint all these (which means optimization requires more advanced paint techniques (to not erase everything), or caching, or scheduler-based paint interruption to skip or change middle frames if newer input/events interrupt).

From a threading perspective, I was surprised that so much happens on the main thread. And that you can have a scheduler running in different threads for compatibility, because most Qt objects ... assume and use a scheduler in their current thread? https://doc.qt.io/qt-5/threads-qobject.html#per-thread-event-loop

The only uses of QThread I could find were from 2014 when synchronization was added back to the main thread from indexer threads. That's fine, but I also see us using QProcess and have to look up settings and such first, etc. I'm thinking the app wasn't structured to be thread-first, and certainly seems to have "UI State" living in a number of places (though this is just an assumption on my part, I haven't investigated the code for this). Now, being C++ and not web JS, we can share memory between threads, and while I don't know exactly how I would implement this (I'm still new to C++), my initial thought was something like Redux -- where the app's state/configuration is loaded in a background thread (it starts initially blank or loaded from a file) and then some kind of signal or inter-thread communication updates the app with the shared memory/shared state.

In React, the assumption is that you pay a heavy cost rendering, so ideally state flows from the outside-in and at any point an object could interrupt and say, "but my part of the state hasn't changed" and not need to re-render. It sounds vaguely like the easiest way to do that might be https://stackoverflow.com/a/17062406/128579 but I don't know what side-effects it might have.

Similarly though, you'd want other threads (at least one other thread) to handle dispatch of state change events, have that thread actually be the one to own/change the shared state, and then trigger the other threads with the updated state for the app.

If you're thinking, okay, but that sounds like a lot of work to go through a giant shared state object, well, that's where selectors come in, they're functions that take the pointer or reference to the giant immutable object/state/thing and chop it down to what's required for that widget or a particular boolean. Selectors are also where sorting, filtering, can take place, and then memoization can be used as a shorthand for building something of an LRU cache for an array of inputs to produce the same output. This is because selectors run so frequently during paint when the inputs to the selectors often don't change as frequently as once every frame/timer/paint interval. Obviously this caching is only used when the overhead of computation outweighs the overhead of caching just the result.

From there, inefficiencies still exist with this data flow -- for instance, you still need a place to put or schedule async events such that your state can always update from user input, but background activities can still occur. You also need a way to automatically react to state changes, or to be more precise, events that change state, that's where in JS-land RxJS ("epics") and Sagas live. The problems they tend to have are complex graphs of dependencies unless you keep them very simple and clean, which isn't a good answer.

Another inefficiency is that a selector -- or a data loading post-event saga/epic -- could load the same data twice, or sort/filter the same data twice. In selectors you might get around this by having one selector call or memoize another selector. In data loading you might try to build caches or smarter logic around when to load fresh data.

A new pattern has been emerging slowly, which is called Relay. The idea is you specify a data model understood at compile time, and instead of specifying selectors, you make queries of your state within your QtWidgets. At compile time, you can then simplify the queries and based on the QtWidget graph (parent/child hierarchy), you can follow the data flow needs for the app, and which screens ("pages" on the web) require which data. Then you can optimize your data fetching for the first page/screen, and afterward loop through the next logical areas of the app and pre-fetch that data next, so load times feel instant. Consider the example at the start, where going backwards is instant but going forwards is not. Well, going backwards doesn't need to load data! Going forwards does! So (assuming I'm interpreting this correctly) there you have the cost to poor data loading, even in a desktop app like this.

I'm describing how Facebook's new website works, but the idea applies here too, maybe. Relay itself keeps a cache of data locally, and has logic built-in around expiring that cache and refreshing data. In our case, the data fetching is likely most noticed if reading from config, or if reading from Sqlite. The data itself doesn't change, but -- and I'm making an assumption here -- the reason we use Sqlite is that we (a) don't want to assume the data all fits in memory (though on my local desktop here, I can load all of the Linux kernel in memory easily, it's a gig or two, right?) and (b) we need something to help us efficiently query the graph data, and outsourcing that to Sqlite is easy enough. What that means is Sqlite is effectively just another set of selectors over the shared state that is a Sqlite database locally. (Or in Relay parlance, selectors are queries)

In our case, we could theoretically listen for file changes on disk, and as files change trigger some form of incremental recompile/reindexing which would then need to trigger cache clearing for existing queries. That's pretty straightforward in Relay, they actually support the concept of "subscribing" to queries, where the server is aware of what queries the client is going to want refreshed data for, and what they might not. In the case where the server is remote, and therefore you can't trust the clients, you can even hard-code a list of queries you support, so as to not receive crazy ones. But in our case, since the client can be assumed to be on a non-UI thread, and the server can be assumed to be local, it doesn't save us any time to have the server contain code to shape the data for the client to the way the client expects, unless we (for whatever reason) want to support multiple clients, perhaps.

This is all theoretical up until now, and based on how React/Relay works on the web. But if there's anything I've learned recently writing non-JS code, it's that non-JS languages can really learn a lot from recent web coding techniques. I'd welcome any thoughts folks have ... I certainly can't say how well any of the above would apply to C++ and I haven't fully thought through which layers can be glossed over or skipped because C++ and shared memory exist, I also haven't fully considered how much can be done at compile time vs runtime, and if so how much we'd want to do at compile time anyway.

[LouisStAmour]

Come to think of it, a prototype port of the app written using Web UI might be fascinating for server/distributed use cases... but I’m straying a bit off topic.

[LouisStAmour]

And React Native is a thing, as is https://github.com/status-im/react-native-desktop-qt though I can’t speak to its quality or performance, as I haven’t investigated.

[LouisStAmour]

Going the other way is possible too, given web assembly. https://webassembly.org/docs/c-and-c++/

[LouisStAmour]

Thinking out loud: Like Bazel and Tableau Prep as examples, you could create a dependency tree for your project’s indexing requirements as step one in creating a graph visualization of your project in SourceGraph. The output has to be something that can be graphed in a standard way in the graph view, linked to source code, etc., ideally though with extensions to display differently for different types of projects. SourceGraph GUI in Qt is itself is a dependency graph of UI, but in a rigid parent-child hierarchy, like React components in a way. Data flows through Bazel, Tableau Prep, Redux and Relay in just one direction, with immutable inputs at each step. We would have two independent pipelines, one for UI updating and the other is for Incremental Indexing, of which full indexing is just a specialized state where we start with no existing state. Well, there’s a second difference, Qt runs in one thread and indexing can be multithreaded. But the GUI could also be multithreaded if you want to co-locate queries or background functions with the ideally heavily optimized or static/simple Qt GUI code intended to execute on the main thread. Ideally you’d keep more of the if statements as assignments or math or in off-thread selectors so the UI has less branching, that said since most inputs wouldn’t change, preventing work early via caching is better still. Again I’m noticing similarities between Bazel and React here...

[LouisStAmour]

Thing is, we have control over the GUI, 100% but we don’t have control over the inputs and compilation steps of projects and so we could have the same problems Bazel has where build systems use outside, untracked state. For example how Sourcetrail downloads Python code in its build process or how CMake find_package can introduce changes or the git repo is checked for a version tag. More common would be MsBuild which uses timestamps rather than file content to determine what files have changed or other uses of timestamp in build processes.

So from that perspective if we want to handle more codebases than Bazel, we’d have to be much more liberal with at least the initial steps of the process and possibly any steps that are given raw user input rather than our own Sourcetrail metadata as inputs. The downside is we have to be indexer agnostic then when it comes to incremental compile, and we can’t write general purpose caching for those steps that given the same inputs it would produce the same outputs. The same problem happens in React with hooks or memoization, so it’s not foreign, but we can’t easily tell people to remove the interactive bits of their compile steps unless we are really liberal in how we parse inputs such that we never have to execute compile steps or build scripts, all we need are raw files to index. And that ... is a lot harder to possibly impossible depending on the language (small talk comes to mind though it doesn’t actually have global state just opaque state...) The problem, thinking about it, is when your app behaves different and in this case since we’re mapping out source code and what it means, we’d actually worry more about generated files which seem to me to be more predictable than build output or the repeatability of linker decisions. I guess what I’m saying is by skipping as much of the build outputs as possible in the indexing, we wouldn’t have to worry about this, our output would hopefully be stable.

It means treating the source code itself as stable immutable inputs and focus more on pattern matching or assigning dynamic outputs as simply metadata or possible outputs of the immutable input source code. But it also means we would need to enforce hermetic steps where given a set of files, any transform process is always guaranteed to produce the same outputs to be used in the next step. For example, variable substitution or adding import headers would always have to produce the same compilable output from the same templated input — rather than using real outside data they would have to consistently fake it. That way we could then map any source ranges from an output back to its input in a consistent way. I’m sure I’m missing something or overcomplicating my explanation, but it’s the only thing I can think of right now. The extra complicated part would be attaching binary analysis or runtime data but we could leave mapping that to the original source code as a problem external to indexing, because it’s solved using source maps and other debug output we don’t need to concern ourselves with or build. Debug output for a build and its original sources would be inputs to further analysis and could be done on top of a graph of a completely different build as long as the symbols extracted to SourceGraph match up enough. To that end, you could show annotations from a production release against source code for a development branch and indicate symbols not found as unindexed.