azavea / pfb-network-connectivity

PFB Bicycle Network Connectivity
Other
39 stars 10 forks source link

Replace CloudWatch logs url in UI #572

Open CloudNiner opened 6 years ago

CloudNiner commented 6 years ago

For a particular job on staging with AWS Batch job id d355ddcf-4628-4ccb-a469-604d3b1277b7, this CloudWatch logs url is printed:

https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logStream:group=/aws/batch/job;prefix=staging-pfb-analysis-run-job--371--c2f908f9/d355ddcf-4628-4ccb-a469-604d3b1277b7;streamFilter=typeLogStreamPrefix

Clicking this leads to an empty view in the CloudWatch logs console. When instead navigating to the correct log via the Batch job details UI, the correct link appears to be:

https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logEventViewer:group=/aws/batch/job;stream=staging-pfb-analysis-run-job/default/9b8d0c32-e4c0-4cbb-aa5d-f5ad3f78c6b9

Unfortunately, the ID at the end of the new, correct link appears to match with nothing we have access to in our current AnalysisJob model or the Batch boto describe_jobs() call. So if we want to present a helpful link to the debug logs, we'll have to do it a different way.

I see two potential (still hacky) solutions.

  1. Use a Batch job detail link instead. This looks like: https://console.aws.amazon.com/batch/home?region=us-east-1#/jobs/queue/arn:aws:batch:us-east-1:950872791630:job-queue~2Fstaging-pfb-analysis-job-queue/job/d355ddcf-4628-4ccb-a469-604d3b1277b7. From this page, the user can scroll to the "View logs for most recent attempt" link at the bottom of the info popout
  2. Construct a new CloudWatch logs url, with custom filtering applied, to get us as close to guessing the correct job log stream as possible. This would likely require a combination of datetime + job queue name filtering.

Both of these are quite brittle. Open to any and all other suggestions using valid public APIs.

KlaasH commented 5 years ago

The batch jobs only show up for a short time after they're run, so while that might be the most direct way in while we're running, it wouldn't work for later reference, which seems like a requirement of these links that we'll be showing permanently on the job detail page in the admin.

I think it makes sense to do something similar to the second option. Though the job UUID doesn't mean anything to Batch, it is mentioned a number of times within the analysis logs. A link that filters for the UUID (quoted) and sets a date window, like this one, will show some of the lines from the correct stream, and from there you can click a link in the "Show in stream" column to get to the full stream. image