lag at end of new entry, increasing with total number of entries

[caver456]

observed when opening a clue dialog - if there are no or very few entries already, the cursor starts blinking in the clue description field immediately, but if there are a few hundred entries already, the cursor delays a second or so before blinking.

Save should be incremental. Investigate why save time seems to be a function of number of entries.

While this lag doesn't slow down the typing of a new entry, it does slow down any processing immediately after a save, and it will increase during the operation which is pretty bad.

This is in the gray area between 'bug' and 'enhancement' - calling it a high priority enhancement for now, since nothing is strictly broken, but it would become unusable after some number of entries.

[caver456]

Currently, save() writes the entire file from the top on every call, which happens after every new entry including the automatic entries like "RADIO LOG SOFTWARE: 'LOCATED A CLUE' button pressed"...

It looks like some options to make this faster / smarter are:

1. on each new entry: open in append mode, write one line, then close
2. open the file on the first entry, and leave it open until the program closes
3. use a different file type, like an actual sql database, maybe with journaling?

1 seems like the lightest-weight, simplest, most robust change. No doubt there is some other file system jujitsu that could be applied here that I'm just not familiar with. 3 seems like a more major overhaul - you'd want to refactor a lot of the code to make use of the fact that it would be a database rather than a simple list (self.radiolog).

So - 1 it is, unless another follower has a different suggestion. 2 or 3 could always be pursued later if there's a real need.

[caver456]

Currently, when the entire radioLog list is written on every call to save, there's no need to keep track of what row(s) need to be written. There are actually no arguments to save, other than 'finalize'.

Going forward, if save is to be incremental, we need to keep track of what row(s) need to be written. This could seemingly be as simple as adding an argument to save.

For robustness, we need to ensure that every row gets written to the file. Is this just being paranoid, or, is there some real mechanism where a row could not get written? A non-fatal traceback in the new entry process which bypassed the call to save would do it.

So, let's have a return value from save: a list of timestamps of rows that were written. Then, the top level timer could check every few seconds for unwritten rows: when a row is added to radioLog, append its timestamp to a list of unwritten timestamps. When a row is written to file, remove its timestamp from that list. This list of unwritten row timestamps should stay empty in the steady state.

[caver456]

Looks like the file save is not the source of the delay! Need to do more profiling / timing, but, the cursor is stuck for a second or two even after the save function appears to be finished. This is visible because the refactored incremental save code calls save() earlier in the newEntry flow - so something in the newEntry flow after save() must be the cause of the delay.

[caver456]

Style sheet strikes again! Turns out the lag (1-2 second period of cursor unresponsiveness in clue description field after opening a clue dialog in a log with hundreds of entries) was due to showTeamTabsMoreButtonIfNeeded which was added for #505 a month ago:

def showTeamTabsMoreButtonIfNeeded(self):
    if self.hiddenTeamTabsList:
        self.ui.tabWidget.setStyleSheet(self.tabWidgetStyleSheetBase+'\nQTabWidget::tab-bar {left:14px;}')
        self.ui.teamTabsMoreButton.setVisible(True)
    else:
        self.ui.teamTabsMoreButton.setVisible(False)
        self.ui.tabWidget.setStyleSheet(self.tabWidgetStyleSheetBase+'\nQTabWidget::tab-bar {left:0px;}')

So - find a way to do this without stylesheets, or at the very least, to only set the stylesheet when the needed state has changed, rather than on every call.

Also - change the title of this issue since the lag is not due to save. It might not even be necessary to do the incremental save.