search

atc-net / atc-dataplatform

A common set of python libraries for DataBricks
https://atc-net.github.io/repository/atc-dataplatform
MIT License
8 stars 3 forks source link

Appinsight reporting #157

Open mrmasterplan opened 1 year ago

mrmasterplan commented 1 year ago

Set up a class that takes as its one parameter the instrumentation key of an azure app insight.

https://docs.microsoft.com/en-us/azure/azure-monitor/app/opencensus-python

You can use the snippets from here to set up the app-insights and get that key:

https://github.com/atc-net/atc-snippets/tree/main/azure-cli/monitor

The new class, maybe named AtcInsightsReporter, should take any arguments that are serializable, to json, and add them as properties to a log entry that is sent to the appinsights. Importantly, the class should use the job introspection available in atc-dataplatform to also add a link to the correct workspace and job-run, so that one can find the way back to the line that caused the log.

The class should be usable as a context manager, where it catches all exceptions and reports them to log analytics. This will allow for developer attention, while also letting the job run to successful completion.

mrmasterplan commented 1 year ago

I have started working on this in the branch feature/insights-reporter if anyone is interested in collaborating.