x/tools/gopls: experiment with an option/capability for server-side file watching

[findleyr]

Philosophically, file watching should be done by the client. In addition to the arguments here, client-side file watching means that the client is responsible for all state transitions on the server, which means the client owns the session lifecycle. This seems like a good idea.

Except that it doesn't work well. Not all clients support file watching, and those that do have spotty and inconsistent support. Sometimes, a file watching pattern that is performant for one client is prohibitively expensive on another, so much so that this is the only place in gopls where we have been forced to specialize behavior based on the client's user agent.

Recently, the cross platform fsnotify package gained support for recursive directory watching (on all platforms except macOS; thanks to @stapelberg for pointing this out). We've also heard from our friends maintaining the Dart LSP that server-side watching has worked well for them.

We should experiment with server-side file watching. In theory, it shouldn't be that hard to wire in:

• Add a new internal gopls setting to control the experiment.
• Hook into server.updateWatchedDirectories to manage glob patterns. I think it's sufficient to check server.Options inside updateWatchedDirectories, and perform state transitions based on the file watching setting in the previous bullet. (register/unregister client side watching as necessary, and manage the lifecycle of the server-side watcher).
• Call server.didModifyFiles on file notifications. See server.DidChangeWatchedFiles. In theory, we could even call DidChangeWatchedFiles on change events, but that is a bit inaccurate as file watcher events are not LSP events.

Once that is done, we should perform a bake off with VS Code's client-side watching. For example: perform a branch switch, and see whether server-side watching or client-side watching gets it right. Also check the CPU cost of file watching.

One possibility is that neither server-side nor client-side watching gets it right. This could be due to gopls bugs around large numbers of simultaneous state changes, or it could be due to bugs in file watching. With server-side watching, we actually have an opportunity to get it right. For example, we could add heuristics by which we validate state with additional polling, thereby enforcing eventual correctness.

CC @adonovan

[gabyhelp]

Similar Issues

• x/tools/gopls: file watchers should be reevaluated after load #60340
• x/tools/gopls: deprecate experimentalWatchedFileDelay #55268
• x/tools/gopls: watch file changes on disk #31553

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[rsc]

This would be really excellent for the long-tail editors. I added an easy "Restart" command to acme-lsp so I can reboot gopls after I've done things like branch switches, since it was too much work to change acme (a C program) to add all the necessary file watching.