linux-nfs / nfsd

Linux kernel source tree
Other
0 stars 0 forks source link

Handle CB_RECALL in stages #15

Closed chucklever closed 7 months ago

chucklever commented 7 months ago

This was bugzilla.linux-nfs.org 354

[Chuck Lever 2020-12-01 15:08:52 UTC] When a client receives an NFS4ERR_DELAY reply, it is responsible for waiting and retrying that request. That can introduce lengthy and unpredictable delays.

The Linux NFS server today handles open conflicts by immediately returning NFS4ERR_DELAY to the opening client, and then sending a CB_RECALL to the client that holds the conflicting delegation. Most commonly, the client holding the delegation responds by returning the delegation immediately. When all parties are on a fast network, this occurs in just a matter of a few milliseconds.

However, the opening client, upon receiving NFS4ERR_DELAY may wait several seconds or longer before sending the OPEN again. This is a needless wait.

To help mitigate this latency, the server could be restructured to send the CB_RECALL first, and then wait just a few milliseconds before replying to the new opener with NFS4ERR_DELAY. That would give the delegation holder an opportunity to resolve the situation quickly and would save a round-trip or two overall.

We could implement this synchronously in the nfsd thread itself, releasing the nfsd and returning NFS4ERR_DELAY only if the delegation holder does not respond immediately.

chucklever commented 7 months ago

[J. Bruce Fields 2020-12-01 15:38:10 UTC] For what it's worth, I believe the current Linux client's OPEN retry delay is controlled by fs/nfs/nfs4proc.c:nfs4_update_delay(), which uses exponential backoff starting with a 100ms delay.

So in the case you describe (immediate delegation return on a fast network), the delay will be less than a second.

Of course, client behavior may vary; but client implementers know that the most likely explanation for a DELAY return to OPEN is a wait for another client to return a delegation, and should be using a retry strategy that works well for that case.

Agreed that it might still be a good idea for the server to handle these small delays internally. I just don't want to overstate the problem.

chucklever commented 7 months ago

[Chuck Lever 2020-12-01 15:56:16 UTC] The client delay used to be longer, so fair enough.

chucklever commented 7 months ago

[Chuck Lever 2023-03-09 15:18:26 UTC] Fixed in v6.1.