mutagen-io / mutagen

Fast file synchronization and network forwarding for remote development
https://mutagen.io
Other
3.47k stars 153 forks source link

sync: use exponential backoff when encountering short-lived loops #517

Closed xenoscopic closed 1 month ago

xenoscopic commented 1 month ago

What does this pull request do and why is it needed?

When we encounter system-level errors (such as ENOSPC), our best bet is to keep the run loop alive and continue retrying synchronization. However, if we try too aggressively, then we end up spinning on the CPU. Our best bet is to use an exponential backoff strategy driven by the number of sequential short-lived synchronization loops encountered. This PR implements such a backoff strategy.

Any other notes for the review process?

It's probably(?) not worth porting this to the forwarding controller because the types of system-level errors that would cause forwarding to fail would typically be caught at endpoint connection (e.g. when a listener is established). In that case, we sleep for the auto-reconnect interval, which means we won't spin. This backoff is designed for those persistent system-level errors that allow synchronization endpoints to connect but then immediately fail the synchronization loop.