Editor constantly freezing in a large Rust codebase

alexkirsz commented 4 weeks ago

Check for existing issues

[X] Completed

Describe the bug / provide steps to reproduce it

I'm trying out Zed again today after a few months of going back to VS Code because of usability issues with Zed.

We have a very large codebase consisting of some 360 Rust crates (+ about the same amount of Swift and Kotlin code). I'm seeing constant freezes in the editor, where the macOS beach ball appears, and I can't click anywhere or do anything while Zed finishes whatever it's doing.

These freezes usually last from 5s to a minute.

I just had one where Zed was hogging 100% CPU, and I had to kill it after 5 minutes of not responding.

Unfortunately, this, alongside the other reported issues, make Zed completely unusable for us. These might be regressions, as I don't recall hitting nearly as many freezes the last time I attempted to switch to Zed.

Environment

Zed: v0.156.1 (Zed) OS: macOS 14.6.1 Memory: 32 GiB Architecture: aarch64

If applicable, add mockups / screenshots to help explain present your vision of the feature

No response

If applicable, attach your Zed.log file to this issue.

Zed.log

These were the last logs before Zed froze. I often see these so I don't think they're related.

2024-10-10T14:47:03.460827+02:00 [WARN] Generic lsp request to rust-analyzer failed: content modified
2024-10-10T14:47:03.462978+02:00 [ERROR] content modified
2024-10-10T14:47:03.563114+02:00 [INFO] Summarizing updated entries took 3.458µs
2024-10-10T14:47:03.676028+02:00 [ERROR] failed to fetch cached embeddings via cloud model

Caused by:
    RPC request GetCachedEmbeddings failed: permission denied
2024-10-10T14:47:05.709405+02:00 [WARN] Generic lsp request to rust-analyzer failed: content modified
2024-10-10T14:47:05.709505+02:00 [WARN] Generic lsp request to rust-analyzer failed: content modified
2024-10-10T14:47:05.7103+02:00 [ERROR] content modified
2024-10-10T14:47:05.974646+02:00 [INFO] Summarizing updated entries took 4.208µs
2024-10-10T14:47:06.12069+02:00 [ERROR] failed to fetch cached embeddings via cloud model

Caused by:
    RPC request GetCachedEmbeddings failed: permission denied
2024-10-10T14:47:11.829188+02:00 [ERROR] no worktree found for diagnostics path "/Users/alexandrekirszenberg/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.38.1/src/macros/select.rs"
2024-10-10T14:47:11.992336+02:00 [ERROR] no worktree found for diagnostics path "/Users/alexandrekirszenberg/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.38.1/src/macros/select.rs"
2024-10-10T14:47:11.993259+02:00 [ERROR] no worktree found for diagnostics path "/Users/alexandrekirszenberg/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.38.1/src/macros/select.rs"

SomeoneToIgnore commented 4 weeks ago

This report is not actionable without profile sample(s) or at least the repository in question + the sequence of actions to reproduce this.

alexkirsz commented 4 weeks ago

@SomeoneToIgnore I'd be happy to provide a profile sample! But I can't share the repository.

Is there a guide on how to best profile Zed on macOS?

SomeoneToIgnore commented 4 weeks ago

Nice.

Then it's the only way, really: you need Instruments (comes with macOS devtools for free) and do similar things you'd do with other profiles of that kind:

open Zed
open Instruments, pick a time profiler:
Select Zed in a somewhat convoluted UI (shows process path on hover):
Press record, switch over to Zed and start doing things that are slow. Ideally, there has to be some pattern you do things; and some "before was good" -> "now it's bad" scenario in that patter, but anything works, really.
After a few seconds of reproducing the issue (does not make any sense to record sampling of a minute of freezing, but one or two beachballs + a few seconds around it seems fine?), switch back to the profiler and hit stop
You'll see something like this (albeit mine is normal and very short, yours will be longer and much more red)

hit cmd-s and save the trace — that is what I need from you (one or multiple; I expect this won't be a miraculous fix so I'll have to come back for you multiple times)
bonus points: check out that heaviest stack trace on the right and try to make a sense of what is exactly within Zed logic code takes the longest (note that multiple threads may have something stuck)

alexkirsz commented 4 weeks ago

Thanks for the guide, I'll go through this in a couple of hours.

Should I build Zed with debug symbols or are you good with the latest release?

SomeoneToIgnore commented 4 weeks ago

Good point, I expect it to be fine but you can also try the --profile release-fast and see if the traces are more readable this way.

gitsmol commented 3 weeks ago

I'm seeing the same thing. It happened for me when I split my monorepo into a workspace with multiple crates. The project isn't huge, 15k loc in 110 files. I'm refactoring a lot of code and now whenever I break something (e.g. I change the function signature in a trait) and enter the diagnostics window, Zed hangs for a while.

I suspect this is related to https://github.com/zed-industries/zed/issues/18658 and I also recall seeing a (fixed) bug in the issue tracker that relates to the order of operations when saving files and Zed/LSP polling for changed files?

alexkirsz commented 3 weeks ago

On my end, apologies for not getting back to this sooner. I've changed quite a few things in my Zed setup, and I'm no longer running into this issue. I will update this thread once I can reproduce the issue consistently.

alexkirsz commented 3 weeks ago

Hey @SomeoneToIgnore! I recorded two runs: one of them has small hangs, and the other has Zed completely frozen for a while (probably forever, but I killed it after 10 minutes).

The heaviest stack trace is <gpui[5720f5226d2db6bd]::app::entity_map::AnyWeakModel>::upgrade.

Here's the trace: https://www.icloud.com/iclouddrive/05bETfxUb_6jBA3BiGfGexrDw#Zed_Hangs

Let me know if you need me to upload it somewhere else. GitHub unfortunately won't accept it as it's 48MB.

EDIT: Recorded a third run with a forever hang: https://www.icloud.com/iclouddrive/027klYrmex898JZFKbbyrq1jw#Zed_Hangs_2

gitsmol commented 3 weeks ago

I'm seeing the same thing, the AnyWeakModel::upgrade call hangs Zed. Not indefinitely, but long enough for me to kill and restart most times. Here's another trace: http://polyprax.nl/pub/zed_hangs_1_20241016_1826.trace.zip

Goffen commented 2 weeks ago

We only have one crate any only a medium sized repository and I get hangs aswell. Rust analyser + Zed borking out

edit: I think for me this was related to disk also running out of space

SomeoneToIgnore commented 2 weeks ago

Thank you, there seems to be a collection of issues, I see at least two, but all somehow related to diagnostics.

Here's one from the long hangs, I won't cover it yet, as most of the comments around wanted to look at AnyWeakModel::upgrade (and it also shows in the 2nd profile) second

So let's look at first

The upgrade is a red herring, it is very visible on the trace but bear in mind that you're looking at some kind of a projection/sample of things, so it lacks details about how much certain things were called in the course of sampling.

upgrade is a weak pointer turned into a strong pointer, not much chance to become slow in one particular case: https://github.com/zed-industries/zed/blob/d95a4f8671acd89b0b640059cafc55eabcf21499/crates/gpui/src/app/entity_map.rs#L489-L514

Such "fundamental" calls being "slow" on the profile usually indicate some hot loop around it. On the same screenshot, above the upgrade call in the hottest trace, there's a ProjectDiagnosticsEditor::new_with_context and by looking at this, there's an only place that's able to produce such stacktraces:

https://github.com/zed-industries/zed/blob/d95a4f8671acd89b0b640059cafc55eabcf21499/crates/diagnostics/src/diagnostics.rs#L195-L200

I think this is a good place to inspect further, and the profile cannot lead us further, so we need to compile Zed and debug (or use dbg!) it during the slow part.

One way to investigate is the pub fn get_by_path and see how many buffers are there on the moment of opening. I do not expect it to be a problem but worth a check: overall, looking up a buffer by path by iterating all buffers open sounds somewhat dangerous if we somehow "leak" them and keep too many items in that collection. Do you have many files open when the problem starts to appear? Or do things start to break only after some time? Then it's definitely worth checking.
The main issue still seems to be the diagnostics: I expect some issues with them catching up during ssh remoting, so will try to investigate things my way.

But that means, when things get slow you have the diagnostics panel open? Are things better without it being open? Do you receive many diagnostics for some files? Or, if you do not have any such panel open, we need to look there.

Assuming that the diagnostics panel is open and, we need to check out how update_excerpts_rx.next()'s counterpart, update_paths_tx is used — maybe it's overly spammed with diagnostics data somehow. One of the usages is enqueue_update_stale_excerpts but that seems not too hot, as being called on a ! button toggle and the initial constructions. But enqueue_update_stale_excerpts is used everywhere, so I would check whether it's overly called somewhere.

alexkirsz commented 2 weeks ago

Yes, I think this is linked to the diagnostics panel in most cases for me. It's not the only diagnostics-panel related issue I'm seeing, it may be related to https://github.com/zed-industries/zed/issues/19019.

I have between 100 and 600 diagnostic issues active at all time, depending on what part of my project I'm currently looking at.

SomeoneToIgnore commented 2 weeks ago

Even when dragging the scrollbar with my mouse, I keep getting sent back to the top

And that sounds like enqueue_update_stale_excerpts in the works, spamming us with the diagnostics updates. So, now we need to track down that spam source and maybe many diagnostics-related issues are solved.

alexkirsz commented 2 weeks ago

How can I help? I don't expect I'll have time to dive into Zed internals to figure out what's going wrong on my end, but I'd be happy to run a test build and try to repro the issue.

SomeoneToIgnore commented 2 weeks ago

Unfortunately, not sure there's another way except debugging around that method or sharing a project so someone else can debug around.

plichard commented 2 weeks ago

Just in case it helps, here's a another trace recorded on linux during what looks like the exact same freeze, while working on a relatively small rust project. I believe I just pressed I to go into insert mode with vim mode.

My project only had 5 warnings at the time, with the diagnostics panel open in another hidden tab. Cannot share the project unfortunately.

I have v0.159.0 installed.

https://share.firefox.dev/4hk75qV

jorikvanveen commented 2 weeks ago

I have (unfortunately) been able to reproduce this issue consistently in this project: https://github.com/jorikvanveen/web-meteen with Zed 0.157.5 on NixOS (full system config can be found on my github page).

It takes about 5 minutes of jumping around and editing random stuff before problems start to occur.

No clue if its related but I am using vim keybinds as well.

I'm on a decently modern Intel laptop with 32GB of ram and plenty of free storage, let me know if more details are needed.

plichard commented 2 weeks ago

I have been using Zed the whole day and I did not have a single freeze since disabling vim mode. It could be a coincidence, but usually I would get one freeze per hour at the very least.

gitsmol commented 1 week ago

@plichard I'm not using vim mode so its probably not the sole cause of the freezes.

SomeoneToIgnore commented 1 week ago

Heads-up, there's https://github.com/zed-industries/zed/issues/18658 which seems to be very close to this issue. That one is being checked by @\osiewicz , so hopefully some improvements will appear in the future.

osiewicz commented 1 day ago

A fix for this issue is available in latest Preview (0.161.0) build. Check it out and see if it fixes the issue for you.

alexkirsz commented 12 hours ago

@osiewicz It seems much better! Filtering only for errors is now instantaneous, where it would previously take a few seconds to register.

Filtering for warnings + errors still takes some time to load, during which this bug occurs: https://github.com/zed-industries/zed/issues/19019

Repro: in a repository with a large number of warnings (365, across many files), on the diagnostics view, I first click on the "Exclude warnings" button to hide warnings. Then, I click on it again. The scrollbar goes a little crazy as many warnings are added one after the other to the diagnostics view, during which time it can't be scrolled properly. Once they're all shown, I can scroll again.

zed-industries / zed