Closed sortie closed 6 years ago
The CCBot/2.0 crawler (https://commoncrawl.org/faq/) requests are dying with HTTP 500. This brings up the user error metric enough that our alerts trigger spuriously. This needs to be fixed.
pub: Handling request: https://pub.dartlang.org/documentation/modern_charts/latest/ pub: Request handler failed Error: Error on line 1, column 26: Invalid HTTP date: expected " GMT". Thu, 19 Jul 2018 01:06:14 MSK ^ Stack: package:http_parser/src/utils.dart 15:5 wrapFormatException package:http_parser/src/http_date.dart 63:10 parseHttpDate package:shelf/src/request.dart 78:29 Request.ifModifiedSince package:pub_dartlang_org/shared/handlers.dart 104:35 isNotModified package:pub_dartlang_org/dartdoc/handlers.dart 103:9 documentationHandler package:pub_dartlang_org/frontend/handlers.dart 97:12 appHandler bin/service/frontend.dart 56:36 _main.<fn>.<fn> package:pub_dartlang_org/shared/handler_helpers.dart 84:25 _userAuthParsingWrapper.<fn> package:pub_dartlang_org/shared/handler_helpers.dart 77:25 _sanitizeRequestWrapper.<fn> package:pub_dartlang_org/shared/handler_helpers.dart 95:27 _redirectToHttpsWrapper.<fn> package:pub_dartlang_org/shared/handler_helpers.dart 43:27 _logRequestWrapper.<fn> package:shelf/shelf_io.dart 88:29 handleRequest package:pub_dartlang_org/shared/handler_helpers.dart 37:5 _handleRequest package:pub_dartlang_org/shared/handler_helpers.dart 32:48 runHandler.<fn> package:appengine/appengine.dart 65:12 runAppEngine.<fn> package:appengine/src/appengine_internal.dart 62:16 runAppEngine.<fn>.<fn>.<fn>
It seems parsing the the date in the HTTP header is failing.
The CCBot/2.0 crawler (https://commoncrawl.org/faq/) requests are dying with HTTP 500. This brings up the user error metric enough that our alerts trigger spuriously. This needs to be fixed.
It seems parsing the the date in the HTTP header is failing.