search

pbek / QOwnNotes

QOwnNotes is a plain-text file notepad and todo-list manager with Markdown support and Nextcloud / ownCloud integration.
https://www.qownnotes.org/
GNU General Public License v2.0
4.64k stars 411 forks source link

[BUG] High system load switching between notes folders #2830

Closed scott-eddy closed 1 year ago

scott-eddy commented 1 year ago

Expected behaviour

Changing between notes folders should happen quickly with little CPU load

Actual behaviour

When changing between folders I have loaded for notes my system experiences high CPU load by the QOwnNotes process and the UI becomes unresponsive. For reference I have the following folders loaded into QOwnNotes: Screen Shot 2023-09-07 at 12 53 49 PM Screen Shot 2023-09-07 at 12 53 53 PM When transitioning folders from "default" to "YPG" I experience the most notable UI freeze and high system load Screen Shot 2023-09-07 at 12 54 30 PM Screen Shot 2023-09-07 at 1 08 42 PM The UI freeze persists for about 3 minutes 35 seconds but after the application becomes responsive again the system load reduces substantially: Screen Shot 2023-09-07 at 1 00 47 PM As a point of reference the "YPG" folder has 3510 untagged notes (these were markdown files I created prior to using QOwnNotes).

Note that switching back to the "default" folder also experiences a ui freeze and high system load but only for 8seconds. This folder only has 1 note

Steps to reproduce

switch between folders and observe a UI delay and high system load, especially present on folders with large amounts of notes

Output from the debug section in the settings dialog

Expand QOwnNotes Debug Information =========================== ## General Info **Current Date**: `Thu Sep 7 13:19:26 2023` **Version**: `23.9.0` **Build date**: `Aug 31 2023` **Build number**: `1051` **Platform**: `macosx` **Operating System**: `macOS 12.6` **Build architecture**: `x86_64` **Current architecture**: `x86_64` **Release**: `GitHub Actions` **Qt Version (build)**: `5.15.2` **Qt Version (runtime)**: `5.15.2` **Portable mode**: `no` **Settings path / key**: `/Users/escott/Library/Preferences/com.pbe.QOwnNotes.plist` **Application database path**: `/Users/escott/Library/Application Support/PBE/QOwnNotes/QOwnNotes.sqlite` **Application arguments**: `/Applications/QOwnNotes.app/Contents/MacOS/QOwnNotes` **Qt Debug**: `0` **Locale (system)**: `en_US` **Locale (interface)**: *empty* **Primary screen resolution**: `2560x1080` **Screen resolution(s)**: `2560x1080, 1920x1080` **Icon theme**: `breeze-dark-qownnotes` **Notes in current note folder**: `3510` **Calendar items**: `0` **Enabled scripts**: `0` ## Server Info **serverUrl**: *empty* **appIsValid**: `yes` **notesPathExists**: *empty* **serverVersion**: *empty* **appVersion**: *empty* ## Spellchecking **Enabled**: `false` **Selected language**: *empty* **Language codes**: *empty* **Language names**: *empty* **Application dictionaries path**: `/Users/escott/Library/Application Support/PBE/QOwnNotes/dicts` ## Note folders **currentNoteFolderId**: `4` ### Note folder `default` **id**: `1` **isCurrent**: `no` **activeTagId**: `-1` **localPath**: `/Users/escott/Notes` **remotePath**: `Notes` **cloudConnectionId**: `1` **isShowSubfolders**: `no` **isUseGit**: `no` **allowDifferentNoteFileName**: `no` **activeNoteSubFolder name**: *empty* **database file**: `/Users/escott/Notes/notes.sqlite` ### Note folder `GroupLead` **id**: `2` **isCurrent**: `no` **activeTagId**: `0` **localPath**: `/Users/escott/GroupLead` **remotePath**: `Notes` **cloudConnectionId**: `1` **isShowSubfolders**: `yes` **isUseGit**: `no` **allowDifferentNoteFileName**: `no` **activeNoteSubFolder name**: *empty* **database file**: `/Users/escott/GroupLead/notes.sqlite` ### Note folder `DivisionLead` **id**: `3` **isCurrent**: `no` **activeTagId**: `-1` **localPath**: `/Users/escott/DivisionLead` **remotePath**: `Notes` **cloudConnectionId**: `1` **isShowSubfolders**: `yes` **isUseGit**: `no` **allowDifferentNoteFileName**: `no` **activeNoteSubFolder name**: *empty* **database file**: `/Users/escott/DivisionLead/notes.sqlite` ### Note folder `YPG` **id**: `4` **isCurrent**: `yes` **activeTagId**: `0` **localPath**: `/Users/escott/YPG` **remotePath**: `Notes` **cloudConnectionId**: `1` **isShowSubfolders**: `yes` **isUseGit**: `no` **allowDifferentNoteFileName**: `no` **activeNoteSubFolder name**: *empty* **database file**: `/Users/escott/YPG/notes.sqlite` ## Cloud connections ### Cloud connection `Default` **id**: `1` **isCurrent**: `yes` **serverUrl**: *empty* **username**: *empty* **accountId**: *empty* ## Enabled scripts ## Settings **646F6E7A_00000000_00000001_6E7A6361_696D6963** (qlonglong): `0` **AKLastCheckInAttemptDate** (QDateTime): `2020-06-22T09:02:00.520` **AKLastCheckInSuccessDate** (QDateTime): `2020-06-22T09:02:34.636` **AKLastEmailListRequestDateKey** (QDateTime): `2021-02-27T21:06:21.281` **AKLastIDMSEnvironment** (qlonglong): `0` **ASIX Elec/ Corp/ AX88179** (QString): `1` **ASIX Elec/ Corp/ AX88772B** (QString): `1` **ATMEL ASF CDC Virtual Com** (QString): `1` **ATMEL ASF CDC and MSC** (QString): `1` **ATMEL AVRISP mkII** (QString): `1` **ATMEL HIDBLDR** (QString): `1` **ActiveNoteHistoryItem** (NoteHistoryItem): `` **Apricorn SATA Wire 6G** (QString): `1` **Atmel Corp/ Atmel-ICE CMSIS-DAP** (QString): `1` **Atmel Corp/ EDBG CMSIS-DAP** (QString): `1` **Barco ClickShare** (QString): `1` **Bose Corporation Bose QC35 II** (QString): `1` **Brother ADS-1500W** (QString): `1` **Brother MFC-J497DW** (QString): `1` **Brother MFC-J5330DW** (QString): `1` **Brother MFC-J6935DW** (QString): `1` **C-Media Electronics Inc/ USB PnP Sound Device** (QString): `1` **CMI USB 10·100·1000 LAN** (QString): `1` **Cable Matters Inc/ Cable Matters USB-C Video Cable** (QString): `1` **CalDigit, Inc/ CalDigit Thunderbolt 3 Audio** (QString): `1` **Camera Manufacturer USB 3/0 Camera** (QString): `1` **Canon MX340 series FAX** (QString): `1` **ContextMenuGesture** (qlonglong): `1` **Country** (QString): `US` **Debug/fakeOldVersionNumber** (bool): `false` **Debug/fileLogging** (bool): `false` **Editor/CurrentSchemaKey** (QString): `EditorColorSchema-cdbf28fc-1ddc-4d13-bb21-6a4043316a2f` **Editor/autoBracketClosing** (bool): `true` **Editor/autoBracketRemoval** (bool): `true` **Editor/disableCursorBlinking** (bool): `false` **Editor/editorWidthInDFMOnly** (bool): `true` **Editor/highlightCurrentLine** (bool): `true` **Editor/indentSize** (qlonglong): `4` **Editor/removeTrailingSpaces** (bool): `false` **Editor/showLineNumbers** (bool): `true` **Editor/useTabIndent** (bool): `false` **Editor/vimMode** (bool): `true` **Elan Digital Systems VUB300-1/20** (QString): `1` **FTDI FT232R USB UART** (QString): `1` **FTDI MaxStream PKG-U** (QString): `1` **FTDI USB <-> Serial** (QString): `1` **FTDI XBIB-U** (QString): `1` **FileDialog/LastPath** (QString): `/Users/escott/YPG` **FileDialog/LastPath-ImportTextFiles** (QString): `/Users/escott/YPG` **Generic Mass Storage** (QString): `1` **Generic USB3/0 Device** (QString): `1` **HP ColorLaserJet MFP M278-M281** (QString): `1` **HP ENVY 4520 series** (QString): `1` **HP ENVY 5000 series** (QString): `1` **HP ENVY Photo 7800 series** (QString): `1` **HP LaserJet MFP M129-M134** (QString): `1` **HP OfficeJet 3830 series** (QString): `1` **HP OfficeJet 4650 series** (QString): `1` **HP OfficeJet 6960** (QString): `1` **HP OfficeJet Pro 6960** (QString): `1` **HP OfficeJet Pro 7740 series** (QString): `1` **HP OfficeJet Pro 8020 series** (QString): `1` **HP OfficeJet Pro 8030 series** (QString): `1` **HP OfficeJet Pro 8710** (QString): `1` **HP OfficeJet Pro 8740** (QString): `1` **HP OfficeJet Pro 9010 series** (QString): `1` **HP PageWide 377dw MFP** (QString): `1` **HP PageWide Pro 577 MFP** (QString): `1` **HP ScanJet Pro 4500 fn1** (QString): `1` **Hewlett Packard MFP M477fdw** (QString): `1` **Identiv SCR3500 A Contact Reader** (QString): `1` **Identive SCR33xx v2/0 USB SC Reader** (QString): `1` **JMicron USB to ATA·ATAPI Brid** (QString): `1` **JMicron USB to ATA·ATAPI Bridge** (QString): `1` **Kingston DT Elite G2** (QString): `1` **LaCie Device** (QString): `1` **LastUpdateCheck** (QDateTime): `2023-09-07T12:33:31.883` **Linux 3/14/70-fslc+gf9325d1 with 2184000/usb RNDIS·Ethernet Gadget** (QString): `1` **LocalTrashDialog/geometry** (QByteArray): `` **Logitech UE Mobile Boombox** (QString): `1` **Logitech USB Receiver** (QString): `1` **MainWindow/geometry** (QByteArray): `` **MainWindow/mainToolBar.iconSize** (qlonglong): `32` **MainWindow/menuBarGeometry** (QByteArray): `` **MainWindow/noteSubFolderTreeWidgetExpandState-2** (QStringList): *empty* **MainWindow/noteTextEdit.code.font** (QString): `Menlo,10,-1,5,50,0,0,0,0,0` **MainWindow/noteTextEdit.font** (QString): `.AppleSystemUIFont,13,-1,5,50,0,0,0,0,0` **MainWindow/noteTextView.code.font** (QString): `Menlo,10,-1,5,50,0,0,0,0,0` **MainWindow/noteTextView.font** (QString): `.AppleSystemUIFont,13,-1,5,50,0,0,0,0,0` **MainWindow/noteTextView.ignoreCodeFontSize** (bool): `true` **MainWindow/noteTextView.refreshDebounceTime** (qlonglong): `600` **MainWindow/noteTextView.rtl** (bool): `false` **MainWindow/noteTextView.underline** (bool): `true` **MainWindow/noteTextView.useEditorStyles** (bool): `true` **MainWindow/noteTextView.useInternalExportStyling** (bool): `true` **MainWindow/showNotesFromAllNoteSubFolders** (bool): `true` **NETGEAR, Inc/ AirCard 791L** (QString): `1` **NoteFolder-1/NoteTabNameList** (QStringList): `TODO 2023-09-07T09.39.35` **NoteFolder-1/NoteTabStickinessList** (QStringList): *empty* **NoteFolder-1/NoteTabSubFolderPathDataList** (QStringList): *empty* **NoteFolder-2/NoteTabNameList** (QStringList): `20230517_AdamRay` **NoteFolder-2/NoteTabStickinessList** (QStringList): *empty* **NoteFolder-2/NoteTabSubFolderPathDataList** (QStringList): *empty* **NoteFolder-3/NoteTabNameList** (QStringList): *empty* **NoteFolder-3/NoteTabStickinessList** (QStringList): *empty* **NoteFolder-3/NoteTabSubFolderPathDataList** (QStringList): *empty* **NoteFolder-4/NoteTabNameList** (QStringList): `20230830_danny_franzy_w_franzy` **NoteFolder-4/NoteTabStickinessList** (QStringList): *empty* **NoteFolder-4/NoteTabSubFolderPathDataList** (QStringList): *empty* **NoteHistory-1** (QVariantList): `` **NoteHistory-2** (QVariantList): `` **NoteHistory-4** (QVariantList): `` **NoteHistoryCurrentIndex-1** (qlonglong): `10` **NoteHistoryCurrentIndex-2** (qlonglong): `4` **NoteHistoryCurrentIndex-4** (qlonglong): `0` **PKSecureElementAvailableFlags** (qlonglong): `1` **PKSecureElementAvailableFlagsByHost** (qlonglong): `3` **PiwikClientId** (QString): `` **Prolific Technology Inc/ USB-Serial Controller D** (QString): `1` **Realtek USB 10·100·1000 LAN** (QString): `1` **SABRENT SABRENT** (QString): `1` **SAMSUNG SAMSUNG_Android** (QString): `1` **SHARP MX-4141N** (QString): `1` **SHARP MX-4141N PPD** (QString): `1` **SHARP MX-4141NPCL_PS** (QString): `1` **SONiX USB DEVICE** (QString): `1` **STMicroelectronics STM32 STLink** (QString): `1` **SanDisk Cruzer** (QString): `1` **SanDisk Extreme** (QString): `1` **SanDisk Ultra** (QString): `1` **ScriptRepositoryDialog/mainSplitterState** (QByteArray): `` **Seagate BUP Slim** (QString): `1` **SearchEngineId** (qlonglong): `2` **SettingsDialog/geometry** (QByteArray): `` **SettingsDialog/mainSplitterState** (QByteArray): `` **Sharp MX-4141N** (QString): `1` **Sharp MX-4141NPCL_PS** (QString): `1` **Sharp SHARP MX-4141N PPD** (QString): `1` **ShowSystemTray** (bool): `false` **StartHidden** (bool): `false` **TagAddDialog/geometry** (QByteArray): `` **Telink Wireless Receiver** (QString): `1` **Texas Instruments MSP Tools Driver** (QString): `1` **Total Phase, Inc/ Beagle USB 480 Protocol Analyzer** (QString): `1` **USB Storage USB Storage** (QString): `1` **WelcomeDialog/geometry** (QByteArray): `` **Western Digital My Passport 0837** (QString): `1` **acceptAllExternalModifications** (bool): `false` **allowNoteEditing** (bool): `true` **allowOnlyOneAppInstance** (bool): `false` **appMetrics/disableAppHeartbeat** (bool): `true` **appMetrics/disableTracking** (bool): `true` **appMetrics/notificationShown** (bool): `true` **autoReadOnlyMode** (bool): `false` **autoReadOnlyModeTimeout** (qlonglong): `30` **automaticNoteFolderDatabaseClosing** (bool): `false` **checkSpelling** (bool): `false` **closeTodoListAfterSave** (bool): `false` **com/blackpixel/versions/ezsRequiredToken** (double): `509302999.416678` **com/versionsapp/v2/ezsRequiredToken** (double): `647359093.662105` **cryptoKey** (qlonglong): `` **currentNoteFolderId** (qlonglong): `4` **currentWorkspace** (QString): `initial` **cursorWidth** (qlonglong): `1` **darkMode** (bool): `true` **darkModeColors** (bool): `true` **darkModeIconTheme** (bool): `true` **darkModeTrayIcon** (bool): `true` **defaultNoteFileExtension** (QString): `md` **demoNotesCreated** (bool): `true` **disableAutomaticUpdateDialog** (bool): `true` **disableSavedSearchesAutoCompletion** (bool): `false` **dockWasInitializedOnce** (bool): `true` **enableNoteTree** (bool): `false` **enableSocketServer** (bool): `true` **enableWebAppSupport** (bool): `false` **externalEditorPath** (QString): *empty* **fullyHighlightedBlockquotes** (bool): `false` **gitCommitInterval** (qlonglong): `30` **gitExecutablePath** (QString): *empty* **gitLogCommand** (QString): *empty* **guiFirstRunInit** (bool): `true` **ignoreAllExternalModifications** (bool): `false` **ignoreAllExternalNoteFolderChanges** (bool): `false` **ignoreNoteSubFolders** (QString): `^\.` **ignoredNoteFiles** (QString): *empty* **imageScaleDown** (bool): `false` **imageScaleDownMaximumHeight** (qlonglong): `1024` **imageScaleDownMaximumWidth** (qlonglong): `1024` **initialLayoutIdentifier** (QString): `full` **insertTimeFormat** (QString): *empty* **interfaceFontSize** (qlonglong): `11` **interfaceLanguage** (QString): *empty* **internalIconTheme** (bool): `false` **itemHeight** (qlonglong): `18` **legacyLinking** (bool): `false` **localTrash/autoCleanupDays** (qlonglong): `30` **localTrash/autoCleanupEnabled** (bool): `true` **localTrash/supportEnabled** (bool): `true` **localTrashSplitterSizes** (QByteArray): `` **markdownHighlightingEnabled** (bool): `true` **motorola XT1635-01** (QString): `1` **motorola moto g(7) power** (QString): `1` **navigationPanelAutoSelect** (bool): `true` **navigationPanelHideSearch** (bool): `false` **networking/ignoreSSLErrors** (bool): `true` **networking/proxyType** (qlonglong): `2` **newNoteAskHeadline** (bool): `false` **noteEditIsCentralWidget** (bool): `true` **noteFileExtensionList** (QStringList): `md, txt` **noteListPreview** (bool): `false` **noteSaveIntervalTime** (qlonglong): `10` **noteSubfoldersPanelDisplayAsFullTree** (bool): `true` **noteSubfoldersPanelHideSearch** (bool): `false` **noteSubfoldersPanelOrder** (qlonglong): `0` **noteSubfoldersPanelShowFullPath** (bool): `false` **noteSubfoldersPanelShowNotesRecursively** (bool): `false` **noteSubfoldersPanelShowRootFolderName** (bool): `true` **noteSubfoldersPanelSort** (qlonglong): `0` **notesPanelOrder** (qlonglong): `0` **notesPanelSort** (qlonglong): `1` **notesPath** (QString): `/Users/escott/YPG` **notifyAllExternalModifications** (bool): `false` **overrideInterfaceFontSize** (bool): `false` **ownCloud/supportEnabled** (bool): `false` **ownCloud/todoCalendarBackend** (qlonglong): `3` **ownCloud/todoCalendarCalDAVPassword** (QString): `` **ownCloud/todoCalendarCalDAVServerUrl** (QString): *empty* **ownCloud/todoCalendarCalDAVUsername** (QString): *empty* **ownCloud/todoCalendarCloudConnectionId** (qlonglong): `1` **ownCloud/todoCalendarDisplayNameList** (QStringList): *empty* **ownCloud/todoCalendarEnabledList** (QStringList): *empty* **ownCloud/todoCalendarEnabledUrlList** (QStringList): *empty* **ownCloud/todoCalendarUrlList** (QStringList): *empty* **ownCloudInfo/appIsValid** (bool): `true` **ownCloudInfo/connectionErrorMessage** (QString): *empty* **ownCloudInfo/notesPathExistsText** (QString): *empty* **ownCloudInfo/serverVersion** (QString): *empty* **recentNoteFolders** (QStringList): `/Users/escott/Notes, /Users/escott/GroupLead, /Users/escott/DivisionLead` **restoreCursorPosition** (bool): `false` **restoreLastNoteAtStartup** (bool): `true` **restoreNoteTabs** (bool): `true` **savedSearches/noteFolder-1** (QStringList): `Note 2023-09-07T09.39.35` **showMatches** (bool): `true` **showMenuBar** (bool): `false` **showStatusBar** (bool): `true` **startInReadOnlyMode** (bool): `false` **systemIconTheme** (bool): `false` **tagWasAddedToNote** (bool): `true` **taggingShowNotesRecursively** (bool): `false` **tagsPanelHideNoteCount** (bool): `false` **tagsPanelHideSearch** (bool): `false` **tagsPanelOrder** (qlonglong): `0` **tagsPanelSort** (qlonglong): `0` **todoCalendarSupport** (bool): `true` **toolbar/1/items** (QStringList): `action_New_note, action_Find_note, action_Remove_note, action_Open_note_in_external_editor, actionShow_local_trash, actionAllow_note_editing, , action_Back_in_note_history, action_Forward_in_note_history, , action_Find_text_in_note, actionReplace_in_current_note, , actionShow_versions, actionShow_trash, actionShare_note, , actionShow_Todo_List` **toolbar/1/name** (QString): `mainToolBar` **toolbar/1/title** (QString): `main toolbar` **toolbar/2/items** (QStringList): `actionFormat_text_bold, actionFormat_text_italic, actionStrike_out_text, actionInsert_code_block, actionInsert_block_quote` **toolbar/2/name** (QString): `formattingToolbar` **toolbar/2/title** (QString): `formatting toolbar` **toolbar/3/items** (QStringList): `actionWorkspaceComboBox, actionStore_as_new_workspace, actionRemove_current_workspace, actionRename_current_workspace, actionSwitch_to_previous_workspace, actionUnlock_panels, , actionToggle_distraction_free_mode, action_Increase_note_text_size, action_Decrease_note_text_size, action_Reset_note_text_size` **toolbar/3/name** (QString): `windowToolbar` **toolbar/3/title** (QString): `window toolbar` **toolbar/4/items** (QStringList): `action_Quit` **toolbar/4/name** (QString): `quitToolbar` **toolbar/4/title** (QString): `quit toolbar` **toolbar/5/items** (QStringList): `action_Encrypt_note, actionEdit_encrypted_note, actionDecrypt_note` **toolbar/5/name** (QString): `encryptionToolbar` **toolbar/5/title** (QString): `encryption toolbar` **toolbar/6/items** (QStringList): `actionInsert_text_link, actionInsert_image, actionInsert_current_time` **toolbar/6/name** (QString): `insertingToolbar` **toolbar/6/title** (QString): `inserting toolbar` **toolbar/size** (qlonglong): `6` **useNoteFolderButtons** (bool): `false` **useUNIXNewline** (bool): `false` **webAppClientService/serverUrl** (QString): `wss://app.qownnotes.org` **webAppClientService/token** (QString): `` **webSocketServerService/bookmarksNoteName** (QString): `Bookmarks` **webSocketServerService/bookmarksTag** (QString): `bookmarks` **webSocketServerService/commandSnippetsNoteName** (QString): `Commands` **webSocketServerService/commandSnippetsTag** (QString): `commands` **webSocketServerService/port** (qlonglong): `22222` **workspace-initial/name** (QString): `Full` **workspace-initial/noteSubFolderDockWidgetVisible** (bool): `true` **workspace-initial/windowState** (QByteArray): `` **workspaces** (QStringList): `initial` ## System environment **USER**: `escott` **__CFBundleIdentifier**: `com.PBE.QOwnNotes` **COMMAND_MODE**: `unix2003` **DISPLAY**: `/private/tmp/com.apple.launchd.Ah8VWW86sK/org.macosforge.xquartz:0` **LOGNAME**: `escott` **PATH**: `/usr/bin:/bin:/usr/sbin:/sbin` **SSH_AUTH_SOCK**: `/private/tmp/com.apple.launchd.QFaeiXd7eT/Listeners` **SHELL**: `/bin/bash` **HOME**: `/Users/escott` **__CF_USER_TEXT_ENCODING**: `0x1F5:0x0:0x0` **TMPDIR**: `/var/folders/qn/5nczc_n110x24p_25m61qtth0000gn/T/` **XPC_SERVICE_NAME**: `application.com.PBE.QOwnNotes.106713919.106714045` **XPC_FLAGS**: `0x0` **QT_LOGGING_RULES**: `qt.network.ssl.warning=false`
scott-eddy commented 1 year ago

This was caused because in my "YPG" folder I had numerous sub-folders that contained e.g. software repositories that didn't have markdown files but also had their own sub-directories for e.g. virtual environments. I (perhaps naively) thought QOwnNotes would be able to rapidly walk these sub-directories to look for markdown files and only load these.

When moving these non markdown notes folders out of the parent-directory transitioning between folders works smoothly as expected.

scott-eddy commented 1 year ago

Going to close this since no action is really necessary. Perhaps adding a warning about sub-directories not containing notes in the documentation would be useful, but certainly not a bug

pbek commented 1 year ago

Traversing a huge amount of files always takes time. 😅 And since attachments and media files are also in the note folder a warning wouldn't make sense.