Lag during typing is causing characters to be re-ordered during auto completion

[spikedom]

Describe the bug

Roam is starting to lag a lot whilst typing. This is exposing a very annoying bug when typing into brackets [[...]] where the characters within the brackets can end up transposed.

It is as if the first few characters are buffered somewhere whilst the auto completion starts up, then are added after any other characters that have been typed. So when typing [[Roam Bugs]] you can end up with [[m BugsRoa]].

Something similar seems to happen when typing shifted (upper case) characters inside the [[...]] where the Shift modifier is sometimes applied to the second [ so that you end up with [{...}] instead of the intended [[...]].

This issue also affects other brackets that trigger completion - whilst typing the above into Roam I ended up with "(er case)upp" instead of the "(upper case)" that I typed.

To Reproduce

Steps to reproduce the behavior:

Type reasonably fast as described above (I am no great typist so it doesn't have to be that fast).

System Information:

• Roam version: 0.7.2-6a07d96e
• Device: Laptop
• OS: Windows 10
• Browser: Chrome

Additional context

The exported database is ~1MB in size.

[filipesilva]

@spikedom also reached out to us via a support ticket and this was caused due to a problem with local graphs that makes them slower over time. This is something we're fixing now, but is taking a while because it requires some architectural changes.

[dgdulaney1]

@filipesilva Do you have any suggestions for those of us currently dealing with this bug? And what would you expect the timeline for speeding up local graphs to look like? I'm a power user and this bug (as well as general slowness) are pretty big nuisances.

Thanks for the hard work.

[spikedom]

Perhaps @filipesilva can confirm, but it looks to me as if there are two issues here. There is a race in the interactions between the editor and the auto complete mechanism that can cause characters to inserted out of order. This is exposed when the system is running slowly due the separate issue with local graphs slowing down over time, and perhaps by external system loads such as Zoom calls.

[spikedom]

If you can live with the loss of information described in https://github.com/Roam-Research/issues/issues/455 and related issues, you can ameliorate the local graph slowdown by exporting it then re-importing it into a new, empty graph. I suspect that what this achieves is to clean up the part of the database that is normally used to hold unsynced changes but which is used to hold the entirety of local graphs.

[dgdulaney1]

Hm, I followed those instructions but the new local graph is still slow and lagging. Thanks for the suggestion, really hope this issue can get fixed!

[dgdulaney1]

Oh and I should've added: my database JSON export is ~5.3 MB

[dgdulaney1]

I don't mean to spam this issue, but I think it's important to share: this bug and the slowness of my graph as of late are serious (dare I say, detrimental) issues for me right now. Frictionless typing, tagging, and bullet navigation were at the heart of what I loved in Roam and lag/slowness are making Roam quite difficult to use.

I'm a Believer, a constant daily user, and am here for the long run, so I'm only sharing this in hopes that improving speed is a priority :D

[filipesilva]

@spikedom can you confirm if the export/import cycle in your case sped up the graph? You are correct that there are the out of order problem is a mixture and a couple of problems taken together.

@dgdulaney1 was there no performance improvement at all after the export/import cycle in your case? I'm currently working on this update to our sync system but it's still probably two weeks off at this point.

[dgdulaney1]

@filipesilva I solved my issue!

I have a single page where all my work TO-DO's go (422 bullets, none particularly lengthy), and when I complete a TO-DO I like to add a #[[today's date]] stamp at the end of the bullet. I'm finding that within any day page that's linked to that large TO-DO page, I experience lag and unwanted cursor movement while typing. When I remove the link between the day page and the large TO-DO page, however, the lag goes away.

I know others have experienced performance issues on pages with hundreds of linked references, and it seems these issues also arises when linking to a very large page.

[spikedom]

@filipesilva - Sorry, I should have said, but yes importing everything into a new graph has brought the performance back up to where it was originally, and the character transposition issue is no longer manifesting itself.

[filipesilva]

@dgdulaney1 that sounds unexpected to me. Just having a ref should not slow anything down.

Let me know if this description of the setup is accurate please:

• make a big page, e.g. Big Page, with a lot of bullets (say, 5000) containing [[TODO]] random text
• on a small page, e.g. Small Page, I enter a bullet that says ref to [[Big Page]]
• when I write more bullets on Small Page, my cursor should start lagged

Does this sound right?

[spikedom]

@spikedom also reached out to us via a support ticket and this was caused due to a problem with local graphs that makes them slower over time. This is something we're fixing now, but is taking a while because it requires some architectural changes.

@filipesilva - is there any update on when the underlying architectural issue will be fixed ?

[filipesilva]

@spikedom the new sync engine that will be used in local graphs is pretty much finished. It's under review now, will be deployed to experimental graphs (the ones we use for API alpha testing) first, then local graphs, then hosted graphs. We hope to weed out bugs via the staggered rollout.

One critical part that's still missing is the data migration from the existing local graph architecture to the new one. Have a plan but immensely worried about anything going wrong, want to make extra sure data is not lost by this process.

[holdenlee]

I also have this problem. The lag also manifests when I press enter and then type - instead of all the text appearing on the next bullet, the first part of the text appears at the end of the current bullet.

[jeremysabath]

Just noticed this for the first time myself this morning. I started noticing it on a large-ish page with a handful of bullets, each 1k-2k words with a handful of tags throughout, as well as couple of block embeds. Page in total probably has 10k words.

I tried going to a brand new page with no links/refs/backlinks and still noticed the lag, making me think that some large sync process was still happening in the background even when I was just making changes to my small page.

[jeremysabath]

You know, I think my issue may only have been somewhat related. I work in the PWA on my iPad most of the time. That was where I experienced the lag. I tried it in the browser on iPad and the lag was gone, so I tried quitting and restarting the PWA and got back to pretty much normal. So take that for what it’s worth but it’s probably not the same issue others described above. #haveyoutriedturningitoffandonagain 😄

[dgdulaney1]

@dgdulaney1 that sounds unexpected to me. Just having a ref should not slow anything down.

Let me know if this description of the setup is accurate please:

• make a big page, e.g. Big Page, with a lot of bullets (say, 5000) containing [[TODO]] random text
• on a small page, e.g. Small Page, I enter a bullet that says ref to [[Big Page]]
• when I write more bullets on Small Page, my cursor should start lagged

Does this sound right?

I finally figured out my issue!

Your description is close, but the problem I'd been running into was e.g. referencing [[Ref1]] somewhere on Big Page, which in turn adds Big Page as a linked ref for [[Ref1]]. Roam then slows down while on the [[Ref1]] page, or any page that is referenced at least once in Big Page .

My Big Page has lots of to-do's, but I was also nesting them in a kanban board. That turned out to be the problem. When I stop using kanban and have just the page with to-do's, Roam runs smoothly.

[wismie]

Hi, I have this issue as well. It happens on my work laptop, but not on my personal PC, which is more powerful. I work on large pages with lots of references and todos. It also happens when I have a query in the sidebar (alas, I am using them a lot to retrieve my to-dos and to revise notes while in meetings).