Pass X-Request-ID from SR UI to SR Manager

[der]

In the current logs the X-Request-ID value isn't carrying from the SR UI to SR Manager.

Verify that the SR UI is definitely passing on the incoming X-Request-ID when it calls the SR Manager. Is so the problem may be in the proxy or SR Manager.

[jonrandahl]

@der I can confirm the rails app is NOT sending the request_id to the SR Manager, will organise this with @tomguilbert to resolve. FYI, I also checked on the dev instance to ensure this was not just my local instance and can confirm the need to adjust the code

TL:DR

For the record in my development instance, using the logs generated via the rails server, I can see that the request_id is indeed in the logs:

{"ts":"2023-06-15T12:30:20.721Z","level":"INFO","request_id":"8c7894c4-3d7d-4da4-8cbe-f1074edbc540","url":"http://localhost:8081/sr-manager/latest-month-available","status":200,"duration":39237,"message":"API GET to 'http://localhost:8081/sr-manager/latest-month-available' succeeded: 200 '2023-04'"}
{"ts":"2023-06-15T12:30:20.764Z","level":"INFO","request_id":"8c7894c4-3d7d-4da4-8cbe-f1074edbc540","url":"http://localhost:8081/sr-manager/report-request?action=show\u0026age=any\u0026aggregate=none\u0026area=CITY+OF+BRISTOL\u0026areaType=county\u0026controller=download_report\u0026period=2023-04\u0026report=avgPrice\u0026utf8=%E2%9C%93","status":201,"duration":42881,"message":"API POST to 'http://localhost:8081/sr-manager/report-request' succeeded: 201 ''"}
{"ts":"2023-06-15T12:30:20.794Z","level":"INFO","request_id":"8c7894c4-3d7d-4da4-8cbe-f1074edbc540","url":"http://localhost:8081/sr-manager/report-request/avgPrice-county-CITY_OF_BRISTOL-by-none-any-2023-04","status":200,"duration":29285,"message":"API GET to 'http://localhost:8081/sr-manager/report-request/avgPrice-county-CITY_OF_BRISTOL-by-none-any-2023-04' succeeded: 200 '{ \n  \"key\" : \"avgPrice-county-CITY_OF_BRISTOL-by-none-any-2023-04\" ,\n  \"status\" : \"Completed\" ,\n  \"url\" : \"http://localhost:8081/sr-manager/report/avgPrice-county-CITY_OF_BRISTOL-by-none-any-2023-04.csv\" ,\n  \"urlXlsx\" : \"http://localhost:8081/sr-manager/report/avgPrice-county-CITY_OF_BRISTOL-by-none-any-2023-04.xlsx\"\n}\n'"}
{"ts":"2023-06-15T12:30:20.804Z","level":"INFO","request_id":"8c7894c4-3d7d-4da4-8cbe-f1074edbc540","method":"GET","path":"/download-report","status":200,"duration":123520}

I can see them in the response headers - in the browser:

HTTP/1.1 304 Not Modified
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
ETag: W/"e9d39e8250547f9f8b3fa4c6937b3439"
Cache-Control: max-age=0, private, must-revalidate
X-Request-Id: 8c7894c4-3d7d-4da4-8cbe-f1074edbc540
X-Runtime: 0.146327

However, they DO NOT appear in the request headers from the app:

GET /download-report?utf8=%E2%9C%93&report=avgPrice&areaType=county&area=CITY+OF+BRISTOL&aggregate=none&period%5B%5D=latest_m&age=any&_partial=true HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9,cy;q=0.8,en-US;q=0.7
Connection: keep-alive
Cookie: seen_cookie_message=yes; hmlr_cookie_policy={"analytics":true}; _ga=GA1.1.1976014349.1686053368; __utmz=111872281.1686303167.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=111872281.1976014349.1686053368.1686306689.1686308855.3; _ga_B4BHXG7Y65=GS1.1.1686570207.3.0.1686570207.0.0.0; _ga_5VYLT7DLRB=GS1.1.1686570207.3.0.1686570207.0.0.0; _gid=GA1.1.78602763.1686747420; _ukhpi_session=Di%2BpCyBj0lX3Wr%2BeTkMKm5mMO2rsF7mfVMISMfUiEOu%2FcsM4lmsm7OkcV3gyUTsFuuXpWpFDEvTHeKiEypPcxI3OPqosCQj4GBgrim%2B%2BaUKXvjWTfoIGMhnWb%2BuhhEx8YwYaAXjJNOOiklCDWPTBJnRUxBx6VFZbkJDcTkdMME%2Fv%2FXS5sYF2uKOgYljaO1jgJ7crLm7kVUYNq7e2slD0Ou5ngCUQ2J2EeJ2bIgOGooLoF%2F13W4gbF589ZzLQk6g6R8h1w3W3lpnPz%2B4jQvvN3pr1u4TyxA%3D%3D--%2BaAHKhBxrVT0WC19--AYlxFLIVr%2FK0DScg9mQXcA%3D%3D; _standard-reports-ui_session=a3F6Z3Mrd3lRcDNjWm9Kd2ZRSm9oaVlsOG8yc3pKQ3BrbkJMbEUzM3BkdnVFa1ZidHF0SFZGdit6MmsrOUhPQldyYUx3Smc0aFFzNWx3YVdjdWhQNWlkTStNSW9yN09yRnFoU3l2RzlkcEt2YWQ4ZWxWSktZbXZMWGxiOHFxQldjOTQ5Y1lBZStoWk5FUXJmeGNLNFdBPT0tLWlsRFNTTGV3MGx5MXZvcmdPWitRZWc9PQ%3D%3D--09097e3ba2b0bb0b51900921b6dec3441e674669; _gat_gtag_UA_21165003_6=1
Host: localhost:3003
If-None-Match: W/"e9d39e8250547f9f8b3fa4c6937b3439"
Referer: http://localhost:3003/download-report?utf8=%E2%9C%93&report=avgPrice&areaType=county&area=CITY+OF+BRISTOL&aggregate=none&period%5B%5D=latest_m&age=any
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36
X-CSRF-Token: aJ5Twzs+RYasNXTwB8m/0oJ5Lxse7oCJzKGoOZJgLE0YeAN5Uuiaaqspadat4ffyTEMO4qWR5YETVaraL+YHRw==
X-Requested-With: XMLHttpRequest
sec-ch-ua: "Not.A/Brand";v="8", "Chromium";v="114", "Google Chrome";v="114"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"

And finally, most definitely NOT in the SR-Manager logs shown locally:

{"ts":"2023-06-15T12:30:20.716Z","version":"1","message":"Request [23] : /sr-manager/latest-month-available : 200","logger_name":"com.epimorphics.standardReports.webapi.LogRequestFilter","thread_name":"http-nio-8080-exec-7","level":"INFO","level_value":20000,"duration":"9000","path":"/sr-manager/latest-month-available","request-id":"23","status":"200"}
{"ts":"2023-06-15T12:30:20.762Z","version":"1","message":"Request [24] : /sr-manager/report-request?action=show&age=any&aggregate=none&area=CITY+OF+BRISTOL&areaType=county&controller=download_report&period=2023-04&report=avgPrice&utf8=%E2%9C%93 : 201","logger_name":"com.epimorphics.standardReports.webapi.LogRequestFilter","thread_name":"http-nio-8080-exec-10","level":"INFO","level_value":20000,"duration":"5000","path":"/sr-manager/report-request?action=show&age=any&aggregate=none&area=CITY+OF+BRISTOL&areaType=county&controller=download_report&period=2023-04&report=avgPrice&utf8=%E2%9C%93","request-id":"24","status":"201"}
{"ts":"2023-06-15T12:30:20.791Z","version":"1","message":"Request [25] : /sr-manager/report-request/avgPrice-county-CITY_OF_BRISTOL-by-none-any-2023-04 : 200","logger_name":"com.epimorphics.standardReports.webapi.LogRequestFilter","thread_name":"http-nio-8080-exec-2","level":"INFO","level_value":20000,"duration":"4000","path":"/sr-manager/report-request/avgPrice-county-CITY_OF_BRISTOL-by-none-any-2023-04","request-id":"25","status":"200"}

For the record, I believe this is because the request via Faraday is setting the headers specifcially as ONLY the ones in the api_request and not passing them on as required:

[app/services/report_manager_api.rb:83]

  def post_to_api(http_url, options, json)
    conn = set_connection_timeout(create_http_connection(http_url))

    conn.post do |req|
      req.headers['Accept'] = 'application/json'
      req.headers['Content-Type'] = 'application/json'
      req.params.merge!(options)
      req.body = json if json
    end
  rescue Faraday::ConnectionFailed => e
    record_failed_connection(http_url, e)
  end

[tomguilbert]

Now this is verified I've created #124 to get the change made to pass the ID from UI to Manager. Suggest we can close this now @der?

[der]

Would have just left the work on this ticket but happy to close this one in favour of the new one if that's easier.