Open despairblue opened 4 years ago
Thanks @despairblue , that does help!
There's a few cases we run into where we continue to 'dirty' the UI in the background - a few cases I've seen:
I need to narrow down add some test cases to help ensure we don't get into this constant-re-render state
+1 here!
I couldn't quite track or metrify the usage, but from hearing the vents on my MPB it's clear that the editor is using a lot of power. I believe it wasn't related to specific usage, just plain file editing.
I haven't figured out a reliable reproduction, but I've managed to hit 100% CPU with only two or three onivim config files open in the editor.
With OniVim open in a large repository I get this (Macos):
In my case, this happens only in TypeScript projects. For example in Go projects (with the VSCode Go extension installed), I haven't had any resource consumption problems whatsoever.
For me it happens regularly with reason projects.
It happens to me almost always (no matter which language) after a certain time. The activity manager in OSX shows an insane number of reactivations (sometimes 10k +), that's when it shoots up in CPU usage and powers up the macbook jet engines...
Ouch, interesting - one thing that might be helpful is a log file. But it's a bit of pain since it seems, in some of those repros, it's only happening after a while. What I'm curious is if there are some events being logged out periodically that are causing us to wake-up / re-render - that might help narrow the issue down.
I'll look into adding a way to get the logs / events at runtime so we can narrow it down. With that switch, once this high CPU usage is observed, we could flip the logging on (ie, via an option in the command palette) and figure out if there is an obvious culprit.
I have just worked a few hours with the editor and I can say this for sure now: It's fine if I never cmd+tab
away. But I literally had it happening just now opening a browser window and writing this message. That switch would be much appreciated, I'd be happy to provide you with anything else you'd need for debugging as well.
This issue is literally the only thing that is a show stopper for starting to advocate the editor to my boss (I can't defend frequent restarts). It's like torture because I learned to love this editor so damn much 😍 (VSCode feels so dang janky now)
/ Edit: Just looked into the docs for creating that log file, I'll try to get a snapshot of it happening if possible
I can't defend frequent restarts
Totally understandable @jsiebern ! Want to get to the bottom of it. Hope you can recommend it to your boss after we get this fixed 😄
Just looked into the docs for creating that log file, I'll try to get a snapshot of it happening if possible
Sounds good! It might not be the most useful at the moment, since it seems like it happens when you Cmd+Tab
- but I'll work on adding this switch on Monday. That way, once we see this happen - we can collect the data and get to the bottom of it.
Hopefully we can get some more info once that's in place - thanks for all the help / information so far
Bad news I think. I'm not able to reproduce it now, the "workaround" to the problem seems to be the performance hit for -f -debug
. At least for me. That'll make it real hard to track down then 😣
The Idle Wake Ups
value that can easily jump to 10k+ (which is usually the point of no return with 25% GPU & 80%+ CPU) without these flags will never go much higher than 4k now.
Sorry for the spam, just trying to add some data points.
Okay, so I think I might have a way to reproduce it sort of reliably.
I worked in a lot of open files (approx 15 I think). I needed to do some work in all of them, just opening them without doing anything didn't work. Then I closed all of them quickly by hitting CMD+W
which resulted in the described behavior. I could repeat this twice now.
I have attached a log file of that session, but as you said, it's doubtful if it's any good. The last parts might be of value though, as I have just closed all files and then watched the activity monitor. I then let the CPU and GPU stuff run for a few seconds and terminated the process. So hopefully it caught something at the end.
Thanks for the log, @jsiebern !
It seems like, towards the end of the log, when the windows are closing - there's some activity with SCM:
[DEBUG] +9ms Oni2.Store.StoreThread : Running effect: Batch:
scm.getOriginalSourceLines
[DEBUG] +0ms Oni2.Store.StoreThread : Effect complete: Batch:
scm.getOriginalSourceLines
[INFO] +18ms Oni2.Store.dispatch : dispatch: (SCM
GotOriginalContent {bufferId = 17;
lines =
This is where we ask the SCM provider for the 'original' content, so that we can show diff markers in the editor.
I'm wondering, for your repro case - what is the working directory? Is it a monorepo with some nested git repos inside? Or a single git repository?
I also saw some syntax highlighting activity:
[INFO] +0ms Oni2.Store.dispatch : dispatch: (Syntax TokensHighlighted {bufferId = 17; tokens = <opaque>})
[DEBUG] +0ms Oni2.Store.dispatch : After: (Syntax TokensHighlighted {bufferId = 17; tokens = <opaque>})
which is a bit curious, as we shouldn't need to do any extra syntax highlighting work for buffers that are unmodified and already open.
Working on a change here to be able to turn on logging at runtime: https://github.com/onivim/oni2/pull/2598
@bryphe Thanks for looking into it. It's a subdirectory of a single git repo, no nested git repos inside. But it happened in basically every directory I have worked in so far - I just found a way to make it fail here with closing the files.
I'll try to catch it with runtime logging as soon as that's available in nightly. Let me know if I can do anything else to provide you with better data.
Ran into this yesterday a lot too (not using oni for daily work yet). For me it mostly happens when I have a couple of files open, then closing some of them and then using ctrl-tab to switch between open files and by this accidentally reopening closed files (as ctrl tab retains a list of every file ever opened). So ctrl-tab seems to be one of the cases triggering it for me.
I can also see that this may be related to the git markers or at least breaking them as they don't update anymore for me, e.g. if I add a line above a change marker then the added line shows up as changed and the changed line below has no marker anymore.
Thanks @jsiebern and @despairblue for the extra details. Still working on getting https://github.com/onivim/oni2/pull/2598 through (CI issues).
One extra data point would be to track down exactly process has the high-CPU usage.
@jsiebern - On OSX - in activity monitor - setting View
-> All Process, Hierarchically
can help:
In particular, this could tell us whether the high CPU usage is in node
(vscode extension host, or the git extension), if it is in the syntax highlight service, or the core editor.
@despairblue - I think we can get similar info with htop
when pressing F5
. It looks like from your original screenshot it's likely the core editor process - but this can help us confirm.
It sounds like in both cases, it's occurring when files are closed (and re-opened). In the meantime, there's a couple of items I'm going to look at to start as I continue working on reproducing:
@bryphe Sorry, no dice with more details (or rather confirmation then I suppose):
But @despairblue is absolutely right, I just had one file open now and did a CTRL+TAB
several times: Spun right out of control.
I'm gonna test this some more by only CMD+P
opening files for a while (and won't close any).
Thanks @jsiebern - it's actually really helpful! Confirms that the CPU usage is in the main editor process (the one that does the rendering), and not the syntax highlight or vscode extension host.
If we can get a debug trace via #2598 :
...should be able to get some details. The PR is merged in master
, but not yet on the portal - should make it in tonight's nightly.
Closing tabs with :q
doesn't seem to trigger it btw.
I also noticed the following behaviour that could be related:
CMD+W
close the fileIt seems that buffers of tabs closed by CMD+W
are kept around in the background when they shouldn't. (Just tested :q!
with the same setup, same behaviour actually)
Waiting on the nightly now to provide you with a trace log 😊
I updated today and used the trace command. When I tail the trace file the only thing it prints out is Oni2.Syntax.Server : Parent still active.
Here's a trace from one of my setups where it goes nuts: onivim2-trace.log.zip
(The beginning will probably be the most interesting as I started the log after being in that high CPU state for a while already as I didn't realize it until the system slowed down significantly)
This may be interesting:
After a while I noticed the CPU going crazy again and started the trace, I tailed the file and it spammed my terminal with this:
[TRACE] [ 0.018s] Revery.UI.Render : BEGIN: Render frame
[TRACE] [ 0.020s] Revery.UI.Render : -- RENDER: pixelRatio: 1.000000 scaleAndZoom: 2.000000 zoom: 2.000000 canvasScaling: 2.000000
[TRACE] [ 0.020s] Revery.UI.Render : -- RENDER: adjustedWidth: 1920 adjustedHeight: 1181
[TRACE] [ 0.025s] Revery.UI.Render : END: Render frame
[TRACE] [ 0.026s] Revery.UI.Render : BEGIN: Render frame
[TRACE] [ 0.035s] Revery.UI.Render : -- RENDER: pixelRatio: 1.000000 scaleAndZoom: 2.000000 zoom: 2.000000 canvasScaling: 2.000000
[TRACE] [ 0.035s] Revery.UI.Render : -- RENDER: adjustedWidth: 1920 adjustedHeight: 1181
[TRACE] [ 0.046s] Revery.UI.Render : END: Render frame
[TRACE] [ 0.046s] Revery.UI.Render : BEGIN: Render frame
[TRACE] [ 0.048s] Revery.UI.Render : -- RENDER: pixelRatio: 1.000000 scaleAndZoom: 2.000000 zoom: 2.000000 canvasScaling: 2.000000
[TRACE] [ 0.048s] Revery.UI.Render : -- RENDER: adjustedWidth: 1920 adjustedHeight: 1181
[TRACE] [ 0.054s] Revery.UI.Render : END: Render frame
[TRACE] [ 0.055s] Revery.UI.Render : BEGIN: Render frame
[TRACE] [ 0.055s] Revery.UI.Render : -- RENDER: pixelRatio: 1.000000 scaleAndZoom: 2.000000 zoom: 2.000000 canvasScaling: 2.000000
[TRACE] [ 0.055s] Revery.UI.Render : -- RENDER: adjustedWidth: 1920 adjustedHeight: 1181
[TRACE] [ 0.061s] Revery.UI.Render : END: Render frame
[TRACE] [ 0.061s] Revery.UI.Render : BEGIN: Render frame
[TRACE] [ 0.062s] Revery.UI.Render : -- RENDER: pixelRatio: 1.000000 scaleAndZoom: 2.000000 zoom: 2.000000 canvasScaling: 2.000000
[TRACE] [ 0.062s] Revery.UI.Render : -- RENDER: adjustedWidth: 1920 adjustedHeight: 1181
[TRACE] [ 0.067s] Revery.UI.Render : END: Render frame
Most of the file is just this. I close oni after 25 seconds and 1.5 mb of logs.
Thanks @despairblue and @jsiebern for the logs! Very helpful.
Indeed, seems like both cases got stuck in this loop:
[TRACE] [ 0.061s] Revery.UI.Render : BEGIN: Render frame
[TRACE] [ 0.062s] Revery.UI.Render : -- RENDER: pixelRatio: 1.000000 scaleAndZoom: 2.000000 zoom: 2.000000 canvasScaling: 2.000000
[TRACE] [ 0.062s] Revery.UI.Render : -- RENDER: adjustedWidth: 1920 adjustedHeight: 1181
[TRACE] [ 0.067s] Revery.UI.Render : END: Render frame
It looks like we're getting stuck in a case where Revery ends up animating non-stop - continually asking for a re-render because it's animating something. Trying to dive in and see why this could be occurring-
Trace log with #2629 in case that helps: onivim2-trace.log.zip
Also did a new one:
Mostly
[TRACE] [14.430s] Revery.Tick : Tick.pump - starting with 3 active tickers.
[TRACE] [14.430s] Revery.Tick : Interval - running timer: Store: Run Effects 1
[TRACE] [14.430s] Revery.Tick : Interval - running timer: Oni2_Editor Apploop 0
[TRACE] [14.431s] Revery.Tick : Interval - running timer: Mouse Hover Timer 75
[TRACE] [14.431s] Revery.Tick : Tick.pump - ending with 3 active tickers.
[TRACE] [14.431s] Revery.UI.Render : BEGIN: Render frame
[TRACE] [14.431s] Revery.UI.Render : -- RENDER: pixelRatio: 1.000000 scaleAndZoom: 2.000000 zoom: 2.000000 canvasScaling: 2.000000
[TRACE] [14.431s] Revery.UI.Render : -- RENDER: adjustedWidth: 1920 adjustedHeight: 1181
[TRACE] [14.443s] Revery.UI.Render : END: Render frame
Thanks @despairblue and @jsiebern ! You're a step ahead of me, I just was thinking I should let you know that #2629 is in :smile: Really appreciate the logs.
The logs are very helpful - it seems the commonality in both your logs is that there is this a runaway mouse hover timer:
[TRACE] [14.431s] Revery.Tick : Interval - running timer: Mouse Hover Timer 75
I'll dive into this further and see why it's being stuck open. The timer will cause a re-render while active, but it shouldn't be active for long - must be a bug there. Will investigate further and keep you posted
@bryphe Thank you so much - after a first few test runs with the current nightly it seems fixed! I'll keep an eye out for it happening again, but so far no issues where I could reproduce it easily before.
Update: I got a light version of it this time. It did not go as crazy and only after doing some editing / closing & opening a lot of files, hopefully this trace will help: onivim2-trace.log.zip
Had it again:
[TRACE] [20.038s] Revery.UI.Render : BEGIN: Render frame
[TRACE] [20.042s] Revery.UI.Render : -- RENDER: pixelRatio: 1.000000 scaleAndZoom: 2.000000 zoom: 2.000000 canvasScaling: 2.000000
[TRACE] [20.042s] Revery.UI.Render : -- RENDER: adjustedWidth: 1920 adjustedHeight: 1181
[TRACE] [20.062s] Revery.UI.Render : END: Render frame
[TRACE] [20.062s] Revery.Tick : Tick.pump - starting with 3 active tickers.
[TRACE] [20.062s] Revery.Tick : Interval - running timer: Editor ScrollY Spring 338
[TRACE] [20.062s] Revery.Tick : Interval - running timer: Store: Run Effects 1
[TRACE] [20.062s] Revery.Tick : Interval - running timer: Oni2_Editor Apploop 0
[TRACE] [20.062s] Revery.Tick : Tick.pump - ending with 3 active tickers.
The mouse ticker is gone though, so one of them was fixed with the PR done :+1:
Just had another one as well going crazy (90+% CPU): onivim2-trace.log.zip
[TRACE] [ 5.070s] Revery.UI.Render : BEGIN: Render frame
[TRACE] [ 5.070s] Revery.UI.Render : -- RENDER: pixelRatio: 1.000000 scaleAndZoom: 1.000000 zoom: 1.000000 canvasScaling: 1.000000
[TRACE] [ 5.070s] Revery.UI.Render : -- RENDER: adjustedWidth: 2560 adjustedHeight: 1338
[TRACE] [ 5.092s] Revery.UI.Render : END: Render frame
[TRACE] [ 5.092s] Revery.Tick : Tick.pump - starting with 3 active tickers.
[TRACE] [ 5.092s] Revery.Tick : Interval - running timer: Editor ScrollY Spring 257
[TRACE] [ 5.092s] Revery.Tick : Interval - running timer: Store: Run Effects 1
[TRACE] [ 5.092s] Revery.Tick : Interval - running timer: Oni2_Editor Apploop 0
[TRACE] [ 5.094s] Revery.Tick : Tick.pump - ending with 3 active tickers.
My bet is that it's the "Editor ScrollY Spring" timer. Might be an idea to try setting editor.smoothScoll
to false
.
Huh, there is editor.smoothScoll
and also experimental.editor.smoothScroll
according to the docs: https://onivim.github.io/docs/configuration/settings
I set both to false, but scrolling is still smooth. :thinking:
@despairblue The experimental
setting no longer exists, it just hasn't been removed from the docs. I submitted #2665 earlier today to fix it.
Setting editor.smoothScroll
to false
works for me. You should see it if you jump between the top and bottom of a long file, for example. When enabled the viewport will scroll gradually, and when disabled it will jump instantly.
I'll make a video tomorrow. For me everything still scrolls in an animated way. 🤷
Video: https://youtu.be/vkSn082cpyw
It's still processing the HD version.
Nevermind me. smoothScoll
can't work. It needs an r
:facepalm:
Works now, I'll use that and see if it changes the CPU consumption, thanks @glennsl
I think @glennsl is right, haven't had the issue for 2 days now while using oni for my day job. :clap:
Same here. I'm also not getting a bug where the buffer is shifted by a line or so once in a while.
Thanks for the updates @despairblue and @jsiebern ! Sounds like we're getting closer. Good call @glennsl on the workaround / troubleshooting!
I'm investigating - trying to figure if it's a general error with the spring or timer hooks - or if it's specific to the spring hook.
Same here. I'm also not getting a bug where the buffer is shifted by a line or so once in a while.
@jsiebern - hmm, like the buffer is rendered incorrectly?
@bryphe I just had another one unfortunately. Between your recent fix and disabling editor.smoothScroll
it has gotten a lot better though (this was the first one today in many hours of work).
[TRACE] [ 0.189s] Revery.Tick : Tick.pump - starting with 8 active tickers.
[TRACE] [ 0.189s] Revery.Tick : Interval - running timer: Notification Animation 323
[TRACE] [ 0.189s] Revery.Tick : Interval - running timer: StatusBar Notification Expirer 322
[TRACE] [ 0.189s] Revery.Tick : Interval - running timer: Oni2_Editor Apploop 0
[TRACE] [ 0.191s] Revery.Tick : Interval - running timer: Store: Run Effects 1
[TRACE] [ 0.191s] Revery.Tick : Interval - running timer: Yank Highlights Animation 108
[TRACE] [ 0.191s] Revery.Tick : Interval - running timer: Yank Highlights Animation 109
[TRACE] [ 0.191s] Revery.Tick : Interval - running timer: StatusBar Background Color Transition 324
[TRACE] [ 0.191s] Revery.Tick : Interval - running timer: StatusBar Foreground Color Transition 325
[TRACE] [ 0.191s] Revery.Tick : Tick.pump - ending with 8 active tickers.
[TRACE] [ 0.191s] Revery.UI.Render : BEGIN: Render frame
[TRACE] [ 0.192s] Revery.UI.Render : -- RENDER: pixelRatio: 2.000000 scaleAndZoom: 1.000000 zoom: 1.000000 canvasScaling: 2.000000
[TRACE] [ 0.192s] Revery.UI.Render : -- RENDER: adjustedWidth: 1792 adjustedHeight: 1017
[TRACE] [ 0.205s] Revery.UI.Render : END: Render frame
The trace file is a little bigger as I was unable to end the editor when trying to hit "Save All" (had to go discard all for it to end).
hmm, like the buffer is rendered incorrectly?
Yeah, this might warrant an issue itself. It did not happen without smoothScroll so far though. Also I have to investigate, if it only happens when editing PHP using the "PHP Intelephense" Plugin. If I can reproduce it I'll record a video and open a separate issue (I only get back to PHP next week at work).
By the way: Thank you for all your effort and your great communication ❤️
It still happens kind of reliably when I CMD+W
close a bunch of tabs very fast unfortunately.
Not sure why these Yank Highlights Animation 1763
timers are still around. As it only happens after working a bit inside the files being closed, this might be related?
[TRACE] [ 0.030s] Revery.UI.Render : BEGIN: Render frame
[TRACE] [ 0.032s] Revery.UI.Render : -- RENDER: pixelRatio: 2.000000 scaleAndZoom: 1.000000 zoom: 1.000000 canvasScaling: 2.000000
[TRACE] [ 0.032s] Revery.UI.Render : -- RENDER: adjustedWidth: 1680 adjustedHeight: 949
[TRACE] [ 0.041s] Revery.UI.Render : END: Render frame
[TRACE] [ 0.041s] Revery.Tick : Tick.pump - starting with 8 active tickers.
[TRACE] [ 0.041s] Revery.Tick : Interval - running timer: Yank Highlights Animation 1763
[TRACE] [ 0.042s] Revery.Tick : Interval - running timer: Yank Highlights Animation 1762
[TRACE] [ 0.042s] Revery.Tick : Interval - running timer: Yank Highlights Animation 1761
[TRACE] [ 0.042s] Revery.Tick : Interval - running timer: Yank Highlights Animation 1760
[TRACE] [ 0.042s] Revery.Tick : Interval - running timer: Yank Highlights Animation 1758
[TRACE] [ 0.042s] Revery.Tick : Interval - running timer: Yank Highlights Animation 1757
[TRACE] [ 0.042s] Revery.Tick : Interval - running timer: Store: Run Effects 1
[TRACE] [ 0.042s] Revery.Tick : Interval - running timer: Oni2_Editor Apploop 0
[TRACE] [ 0.045s] Revery.Tick : Tick.pump - ending with 8 active tickers.
Yeah, this might warrant an issue itself. It did not happen without smoothScroll so far though. Also I have to investigate, if it only happens when editing PHP using the "PHP Intelephense" Plugin. If I can reproduce it I'll record a video and open a separate issue (I only get back to PHP next week at work).
An issue would be great for it (and the video) :+1: Thanks for all the help! I wonder if the PHP Intelephense plugin is modifying the file externally, or something. It's interesting that it only reproduces with smooth scroll on.
Not sure why these Yank Highlights Animation 1763 timers are still around. As it only happens after working a bit inside the files being closed, this might be related?
Ya, this is helpful for sure, and does seem related - it confirms my suspicion that the problem is more general.
There's actually a descriptive comment here: https://github.com/revery-ui/revery/blob/833d86dfc8b718441702b23e9d73a4a84c7154cf/src/UI_Hooks/Timer.re#L16 - but it's essentially the same issue. Seems like anything using a timer-hook is susceptible to this (including the scroll spring, color transition, yank highlights).
I'm evaluating some options still - I think the hook logic is tricky to get right, so for these animations, I may experiment with moving to an elm-subscription-model, like: https://package.elm-lang.org/packages/mdgriffith/elm-style-animation/latest/ - this would be more robust, since it wouldn't depend on tricky timing dependencies in the view or mount/unmount of components.
Bottom line though is I'm working on a holistic fix to address this class of issue - whether it's through the timer hook, or switching to a different, more robust animation model.
By the way: Thank you for all your effort and your great communication heart
Thank you for all the help troubleshooting and grabbing logs, really appreciate it! Been incredibly helpful
Chipping away at these:
I think once we get through these - should fix the cases seen in the logs so far. And then, beyond that, migrating the remainder of the hook-based timers to model-based subscriptions. Hopefully getting closer!
I'm running latest nightly from today, which includes the fix for the smooth scroll. So I enabled it again, but I'm seeing this issue happening a lot again now:
[TRACE] [17.151s] Revery.UI.Render : BEGIN: Render frame
[TRACE] [17.153s] Revery.UI.Render : -- RENDER: pixelRatio: 1.000000 scaleAndZoom: 2.000000 zoom: 2.000000 canvasScaling: 2.000000
[TRACE] [17.153s] Revery.UI.Render : -- RENDER: adjustedWidth: 1920 adjustedHeight: 1061
[TRACE] [17.164s] Revery.UI.Render : END: Render frame
[TRACE] [17.164s] Revery.Tick : Tick.pump - starting with 4 active tickers.
[TRACE] [17.164s] Revery.Tick : Interval - running timer: Oni2_Editor Apploop 0
[TRACE] [17.164s] Revery.Tick : Interval - running timer: Store: Run Effects 3
[TRACE] [17.164s] Revery.Tick : Interval - running timer: Notification Animation 470
[TRACE] [17.164s] Revery.Tick : Interval - running timer: Notification Animation 471
[TRACE] [17.164s] Revery.Tick : Tick.pump - ending with 4 active tickers.
Hope that helps. Let me know if you need the full log.
I try disabling the smooth scroll again for now.
@bryphe It has gotten so much better, thank you. It just happened again, so I wanted to provide a trace in case there is something we missed.
Unrelated maybe: It looks like there is something funky going on at the end from line 5126
onward. This is directly after saving. It seems that every single char of that buffer has been processed in some way, just want to confirm that this is normal or maybe just for debugging when tracing is active?
Fixed a few more places with potentially hanging timers in #3603
These are the remaining areas I need to fix as next step:
I could figure out which of the two oni process this is and got a callstack for it:
Not sure if this helps.