Integrate with the mypy daemon

[gatesn]

@gvanrossum creating a separate issue to discuss this.

I've poked around mypy a little and I'm very pleased at the sight of the fscache module. The reason pyls doesn't integrate with pylint is because we have no way of telling pylint to read from our workspace (in-memory view of what's in the client's buffers).

I imagine it would be pretty easy to patch the FileSystemCache class to read from our workspace. The downsides of this approach are:

• FileSystemCache isn't public API so it could change at any time.
• We can't easily patch something inside the mypy daemon. And even if we can, we need to make it talk back to pyls.

Another approach might be to run the mypy daemon within the pyls process. The daemon's lifetime is then tied to that of your client which seems quite reasonable, particularly if we make use of the remote caching feature. I might give this a go.

[gvanrossum]

The mypy internals are fraught with hazards, and we prefer not to expose them. But perhaps so is your workspace? We also have reasons to keep the mypy daemon process independent (the code base we work with has 10,000 Python files and totals over 5M lines, and it takes the daemon a while to start up). Perhaps we can add RPC calls to dmypy (the client) to tell the daemon (server) process to update the contents of a file? We'd be happy to extend the protocol between client and server.

[gatesn]

That seems like a better approach. However it does mean that the daemon needs to be able to store and flush out session state for each client. I'd propose mirroring a small part of the LSP and have the daemon support commands equivalent to didOpen, didChange and didClose.

These translate to:

• didOpen - client now controls the source-of-truth of the document and it shouldn't be read from disk
• didChange - client changes the contents of the document (could merge this with didOpen)
• didClose - source-of-truth now lives on disk.

Do you ever see a world where the mypy daemon might want to accept multiple connections concurrently?

[gvanrossum]

Yes, that API makes sense to me. I don't expect the daemon to handle multiple connections concurrently, the code just isn't written for thread-safety.