DataBiosphere / azul

Metadata indexer and query service used for AnVIL, HCA, LungMAP, and CGP
Apache License 2.0
7 stars 2 forks source link

Combine logged request query and headers into one JSON literal #4868

Closed dsotirho-ucsc closed 1 year ago

dsotirho-ucsc commented 1 year ago

Currently requests are logged with two separate JSON literals, one for the query parameters, and one for the request headers. Combine these two JSON literals into one nested JSON structure, such that:

[INFO] 2022-12-31T23:36:50.891Z c9300ddf-07c3-4b21-abe5-5c02297b5997 azul.chalice Received HEAD request for '/index/projects', with query {"size": "1"} and headers {"accept": "/", "accept-encoding": "gzip, deflate", "cloudfront-forwarded-proto": "https", "cloudfront-is-desktop-viewer": "true", "cloudfront-is-mobile-viewer": "false", "cloudfront-is-smarttv-viewer": "false", "cloudfront-is-tablet-viewer": "false", "cloudfront-viewer-asn": "14618", "cloudfront-viewer-country": "US", "host": "service.dev.singlecell.gi.ucsc.edu", "user-agent": "python-requests/2.28.1", "via": "1.1 3042bd56e0ca0a7910df89f6b5e95e9e.cloudfront.net (CloudFront)", "x-amz-cf-id": "lcIU9wetF_-YknujlxxQUIZd-w1jrZ2l7NptE4QcYmXZU6LRrOmkmA==", "x-amzn-trace-id": "Root=1-63b0c792-601592823a75396a6e5f4371", "x-forwarded-for": "52.20.215.168, 15.158.50.144", "x-forwarded-port": "443", "x-forwarded-proto": "https"}.

would become:

[INFO] 2022-12-31T23:36:50.891Z c9300ddf-07c3-4b21-abe5-5c02297b5997 azul.chalice Received HEAD request for '/index/projects', with {"query": {"size": "1"}, "headers": {"accept": "/", "accept-encoding": "gzip, deflate", "cloudfront-forwarded-proto": "https", "cloudfront-is-desktop-viewer": "true", "cloudfront-is-mobile-viewer": "false", "cloudfront-is-smarttv-viewer": "false", "cloudfront-is-tablet-viewer": "false", "cloudfront-viewer-asn": "14618", "cloudfront-viewer-country": "US", "host": "service.dev.singlecell.gi.ucsc.edu", "user-agent": "python-requests/2.28.1", "via": "1.1 3042bd56e0ca0a7910df89f6b5e95e9e.cloudfront.net (CloudFront)", "x-amz-cf-id": "lcIU9wetF_-YknujlxxQUIZd-w1jrZ2l7NptE4QcYmXZU6LRrOmkmA==", "x-amzn-trace-id": "Root=1-63b0c792-601592823a75396a6e5f4371", "x-forwarded-for": "52.20.215.168, 15.158.50.144", "x-forwarded-port": "443", "x-forwarded-proto": "https"}}.


hannes-ucsc commented 1 year ago

For demo, show new log message. Show that Cloudwatch parses all headers and query parameters into separate fields that can be filtered on.