microsoft / mlops-promptflow-prompt

MLOps for PromptFlow SDK
MIT License
32 stars 2 forks source link

Add otel collector and auto instrumentation to fastapi in k8s #102

Closed joshuaphelpsms closed 5 months ago

joshuaphelpsms commented 5 months ago

Description

This adds:

I verified that logs, traces and metrics were landing in AppInsights.

closes #88

A couple of notes:

  1. With the current way that we deploy the fastapi app, it doesn't redeploy when we call kubectl apply. I believe this is due to having no changes in the yaml. Even though I set imagePullPolicy to Always, the redeployment is not triggered. I needed to delete the pod for it to come up with the latest version.
  2. Running this pipeline for the first time will fail since the namespace basic-flow-app needs to exist before creating the secrets. We may want to modify this so the namespace gets created first.

To test:

  1. connect to aks cluster az aks get-credentials --resource-group $MY_RESOURCE_GROUP_NAME --name $MY_AKS_CLUSTER_NAME
  2. port-forward the service kubectl port-forward svc/basic-flow-app-service 8080:8080 -n basic-flow-app
  3. go to browser and hit one of the endpoints http://localhost:8080/class_basic_flow?entity_type=people%27s+full+name&text=The+novel+The+Great+Gatsby+was+written+by+F+Scott+Fitzgerald.
  4. Go to AppInsights instance and click the Transaction search tab. It will take around a minute for the telemetry to show up