microsoft / azure-container-apps

Roadmap and issues for Azure Container Apps
MIT License
362 stars 29 forks source link

Feature Request: Log streaming for manually triggered container app jobs #771

Open MatthewWilkes opened 1 year ago

MatthewWilkes commented 1 year ago

Is your feature request related to a problem? Please describe.
The command-line usage of executing a job is suboptimal, both for direct usage and in a pipeline. Wishing to run a job, then view the logs on completion looks like this in a terminal sessions:

$ az containerapp job start --ids "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-name/providers/Microsoft.App/jobs/job-name" --command "drush cr"
Command group 'containerapp job' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
{
  "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-name/providers/Microsoft.App/jobs/job-name/executions/job-name-qpf0rxs",
  "name": "job-name-qpf0rxs",
  "resourceGroup": "rg-name"
}

$ az containerapp job execution show -g rg-name -n job-name --job-execution-name job-name-qpf0rxs --subscription "Subscription Name" --query "properties.status"
Command group 'containerapp job' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
"Running"

$ az containerapp job execution show -g rg-name -n job-name --job-execution-name job-name-qpf0rxs --subscription "Subscription Name" --query "properties.status"
Command group 'containerapp job' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
"Succeeded"

$ az monitor log-analytics query --workspace 11111111-1111-1111-1111-1111111111111  --analytics-query "ContainerAppConsoleLogs_CL | where ContainerGroupName_s startswith 'job-name-qpf0rxs' | sort by TimeGenerated asc | project TimeGenerated, Log_s" --subscription "Subscription Name" --output table
Log_s                              TableName      TimeGenerated
---------------------------------  -------------  ----------------------------
[success] Cache rebuild complete.  PrimaryResult  2023-06-02T15:21:01.8171803Z

There are a few difficulties here. Firstly, you can use the --ids format to start a job execution, but not to show an execution. Then, you cannot await job completion, merely query the status. Finally, once a job has completed, you need the id of the log analytics workspace and a Kusto query in order to view what happened.

Describe the solution you'd like.
I'd love to be able to await a manually triggered job's completion, streaming logs as it runs. This would allow integration of long-running jobs into CI/CD pipelines, showing execution progress as it completes.

Less optimally, I'd like to be able to attach to a job using a subcommand of execution, similarly blocking until completion and streaming logs.

Describe alternatives you've considered.
It would be possible to build a wrapper around the az cli to deliver this, however it would require polling of the law, running multiple queries with filtering, then deduplicating logs to show on screen.

Additional context.
N/A

JustinGrote commented 1 week ago

I've been working on a PowerShell watcher. Major issue is that container logs tend to show up with the exact same timestamp down to the microsecond so they can get jumbled. https://gist.github.com/JustinGrote/50b0281400755d46cff214c8336a2b9e