I'm assuming WebException is thrown when there is a 404 (or other non-200 status code). You don't want to retry if there's a non 200 status code. You want to send the response back to the crawler so they can update their index.
Retrying is unnecessary because crawlers handle timeouts in their own way and retrying isn't going to help, especially if you are retrying on a non-200 status code.
I don't think this provides any benefit unless you are seeing issues with the Prerender server?
I'm assuming
WebExceptionis thrown when there is a 404 (or other non-200 status code). You don't want to retry if there's a non 200 status code. You want to send the response back to the crawler so they can update their index.Retrying is unnecessary because crawlers handle timeouts in their own way and retrying isn't going to help, especially if you are retrying on a non-200 status code.
I don't think this provides any benefit unless you are seeing issues with the Prerender server?