Navigator Stutters OneNote interface on pages with a large number of titles

[jasonjac2]

Problem to Solve

As always, thank you for your work on the add-in.

When Navigator is up and you navigate to a page with a large number of titles, it causes OneNote to stutter (and anecdotally increases OneNote hangs)..

My problem is that my "monthly" pages have a heading for each day, plus 5 sub headings: Must; should; nice; extra; log, so by the end of the month, that is 31*6 headings and 40+ "page downs". Combine this with having about 5+ instances of onenote up and flicking between them rapidly throughout the day, causes OneNote to perpetually be stuttering and so I don't use Navigate as much as I would like to and can't leave it as a "always up" window.

A clear and concise description of what the bug is. It appears that when you goto a page there is a process whereby:

• Navigator starts to refresh - not immediately
• a few seconds later maybe 3 - the OneNote interface hangs for 1 - 3 seconds and anything you type is delayed appearing on the screen.
• then the UI becomes responsive for a couple of seconds and the screen catches up
• then another brief stutter while the actual content of the navigator screen is populated

Steps to reproduce the behaviour: I have sent you a video "OneMore Stutter with navigator" (to follow in the next few mins)

Proposed Behavior

I assume that this is not a problem a lot of people are having it and unless there is a very clear optimisation that can occur in the getting; parsing; list generation, then the next best thing is to be able to do one of the following:

• Associate the navigator to one of the instances of OneNote - then you intend to use it with that one and it will only interfere when you navigate with that instance of OneNote.
• Have a setting where it doesn't auto refresh and there is a button and keyboard shortcut for it to refresh.

Bonus: Allow filtering in the window :-).

[stevencohn]

Note to self: keep everything down to here pre-invoke to stay off the UI thread

@jasonjac2 The navigator is not tied to any OneNote window on purpose. Instead, it polls the Navigator.json file on a regular interval, which can be changed in Settings. It defaults to two seconds and can be set 1..60. The hot path is when it tries to build out the hyperlinks for display; that API back to OneNote is slower than it should be, but that's also internal to OneNote's API. So while I may not be able to increase the speed very much, I should have a couple of alternative that don't lock up the UI thread... Will take a look.

[jasonjac2]

"The hot path is when it tries to build out the hyperlinks for display; that API back to OneNote is slower than it should be, but that's also internal to OneNote's API." So does this mean that it building the "Headings" for the current page - which is the bit that takes the time (top list) there are n call backs to the ON API (n = number of headings?). Could it delay this until you actually click one of them? I can't work out whether it is the size of the page OR the number of headings - I could test this with a massive page with a few headings vs a small page with loads of headings.

[jasonjac2]

I assume: await using var one = new OneNote(); var heading = (Models.Heading)label.Tag; await one.NavigateTo(heading.Link); is expensive and is called for a squillion headings that won't get clicked

[stevencohn]

No, that's where navigation happens when you click on a heading to force OneNote to move to that page. That's pretty quick.

Building the hyperlinks starts here where it loads the page and collects the headings. That part is quick.

The slowness is here where gets the hyperlink for each heading. This line is wrapper of the OneNote API but it's that wrapper OneNote API that is slow. Nothing can be done about speeding up the API.

But the current code has that running on the UI thread for OneMore which is likely causing responsiveness issues in OneNote as well. My theory is that I can keep this in a background thread until all hyperlinks are discovered and the switch over to the UI thread to update the Navigator display. Worth a shot anyway.

[jasonjac2]

"hyperlinks are discovered and" Ignore me if you're busy, but just trying to get it into my head: For each heading...... there is a callback into ON and those are expensive? If so, can't you defer those until one is actually clicked?

[jasonjac2]

I should really compile the blinking code and learn rather than just ask away!

[stevencohn]

can't you defer those until one is actually clicked?

Um, er, well, that would be smart, wouldn't it?

[jasonjac2]

I can't tell if you are mocking me :-).

[stevencohn]

Mocking myself!

[stevencohn]

Result

• On a page with 1000 headings
• Saved 84% - from ~7.6s to ~1.6s

  Baseline

• Time to load and display page with 1000 headings in Navigator averaged ~7.6s
• UI update was bulk of time at ~6.1s, not loading the page headings or discovering URLs

  After changes <--

• Load all headings without URL, see GetHeadings line. URL lookup deferred until linked is clicked
• UI update suspended until flow control is fully populated
• UI update takes ~1.60s
• Total time to load and display avg ~1.61 seconds

  04|05:17:03.285+ ..LoadPageHeadings [Page 1000 Headings]
  04|05:17:03.300+ ..00:00.00 GetHeadings <---
  04|05:17:03.300+ ..00:00.00 suspending layout
  04|05:17:04.964+ ..00:01.67 resumed layout
  04|05:17:04.964+ ..00:01.67 LoadPageHeadings done

  Up-Front Loading of URLs

• Loading all heading with URLs, GetHeadings line, takes about 0.81s
• UI update unchanged at ~1.6.0s
• Total time to load and display avg ~2.51s
• So deferring the lookup of each URL was indeed a good idea

  10|05:17:53.635+ ..LoadPageHeadings [Page 1000 Headings]
  10|05:17:54.481+ ..00:00.85 GetHeadings <---
  10|05:17:54.481+ ..00:00.85 suspending layout
  10|05:17:56.110+ ..00:02.47 resumed layout
  10|05:17:56.110+ ..00:02.47 LoadPageHeadings done

[jasonjac2]

Wow, so thorough. Thank you for that.

[jasonjac2]

What are you hoping to cram in before the next release and any idea of when? Have you thought about doing a RC and getting help with the docs?

[stevencohn]

I generally keep an eye on these three filters day-to-day

• Ready ffor next release
• Open defects
• Open feature requests

I try to keep the docs up to date with every ticket. Remember, it's a local notebook that I only publish to the Wiki with each release, so there's a lot of work happening in the background.

There is more work to complete in #1335, which admittedly is quite large since I've used it to finally address some long-awaited refactoring; that, in turn, uncovered some other hidden problems that need solving before a release.

So, next couple of weeks or sooner if I can?!

Finally, the following info section will be published as part of the vNext release notes. These are ideas/thoughts/known issues that haven't become tickets yet. Here's a preview:

Known Issues and Proposed Updates

🐜Reduced an issue where Command/Quick Palette dialogs would cause the OneNote window to flicker; this changes the opening behavior of the Palette dialogs, hiding the command list until you start typing, #1422. However, the flicker still randomly occurs. 🐜 SearchAndReplace doesn't replace the currently highlighted or selected word/phrase; sometimes it doesn't replace other candidates either 🐜Replaying a command using Ctrl+Shift+R is done in a single thread, breaking progress.RunModeless; see CreatePagesCommand 🐜Select Similar Styles will find differences in colors Auto vs #000000 - make it more fuzzy? 🐜PasteText should convert CRLF to hard-breaks, currently creates soft-breaks 💡Command to convert friendly URLs to raw URLs; References menu 💡Paginate Hashtag search results 💡Move Navigator to DB, paginate history panel, provide import/export 💡Move Reminders to DB, option in dialog and on-page link to delete completed reminders 💡Github PR action to pull ticket# prefix from branch name, e.g. 1234-do-something-cool

[jasonjac2]

Excellent, thanks for the information.
"So, next couple of weeks or sooner if I can?!" amazing. On the "move to DB..." items, if you do that, will filter be easier to implement later? If you have a workaround for #1523 that would be great (rather than just delete them all and start again).

[stevencohn]

DB - yes, filtering, pagination, and helps cross-machine mapping to pages where we know there is problem in that OneNote is free to assign different IDs to things which breaks, well, everything!

Moving Favorites there will have the same benefit.