Azure OpenAI Integration

[SubhrataK]

The Azure OpenAI Service grants users REST API access to OpenAI's robust language models, encompassing the GPT-4, GPT-4 Turbo with Vision, GPT-3.5-Turbo, and Embeddings model series. This service equips customers with cutting-edge language AI, featuring OpenAI models such as GPT-4, GPT-3, Codex, DALL-E, Whisper, and text-to-speech models, all within the secure and enterprise-centric environment of Azure. Collaboratively developed with OpenAI, Azure OpenAI ensures seamless compatibility and a smooth transition between APIs.

By leveraging Azure OpenAI, customers benefit from the security features inherent in Microsoft Azure while utilizing the same powerful models as those offered by OpenAI. The service provides additional advantages, including private networking, regional availability, and responsible AI content filtering. Azure OpenAI is part of Azure AI services, which uses Azure Monitor.

This integration shall capture the logs and metrics for Azure OpenAI.

Metrics:

• Azure OpenAI Requests - Total number of calls made to the Azure OpenAI API over a period of time.
• Generated Completion Tokens - Number of generated tokens (output) from an OpenAI model.
• Processed FineTuned Training Hours - Number of Training Hours Processed on an OpenAI FineTuned Model
• Processed Inference Tokens - Number of inference tokens processed by an OpenAI model. Calculated as prompt tokens (input) + generated tokens.
• Processed Input Tokens - Total number of prompt tokens (input) processed on an OpenAI model.
• Provision-managed Utilization - Provision-managed utilization is the utilization percentage for a given provisioned-managed deployment.

More metrics to be added. Reference: https://learn.microsoft.com/en-us/azure/ai-services/openai/overview

### Tasks
- [ ] https://github.com/elastic/integrations/issues/9376
- [ ] https://github.com/elastic/integrations/issues/9378
- [ ] https://github.com/elastic/integrations/issues/9547
- [ ] https://github.com/elastic/integrations/pull/9706

[muthu-mps]

The Azure OpenAI integration captures the metrics and logs from the Azure OpenAI service.

Metrics

• The resource API is used to capture the metrics from the Azure OpenAI services.
  • Resource Type: Microsoft.CognitiveServices/accounts
• The resource type definition is added to the streams.hbs file which includes the metrics to be captured from the cognitive services.
• The metrics defined in the above description will be captured along with few additional metrics which is useful for monitoring.
• The metric dimensions are included and verified.

Logs

• We are considering the Audit and the Request Response category logs from the Azure OpenAI services.
• Since the Audit log captures only the ListKey logs. It makes sense to add both of the log categories in a single data stream.
• Enhance the ingest pipeline to process the properties object from the logs as individual fields.

Azure openAI Monitoring

[muthu-mps]

@zmoog - I have done the analysis and currently started working on the implementation. Captured the metrics and log collection methods. Created different issues one for logs and other for metrics which has more specific details about the data processing.

After discussing with you on the dimensions, I have updated the issue on how the document would look like once we have the dimensions for the metrics. Please feel free to add your suggestions if any.

[muthu-mps]

• This integration has initially developed with the default mappings for the logs data stream. Recently there is an ongoing effort from security team to make LLM fields adopt OTEL standard. Based on which the AWS Bedrock integration has been developed. You can find more details regarding the LLM field standardisation here.
• This change is more aligned with the Azure OpenAI integrations and we are exploring the LLM fields to make the appropriate changes for logging data stream.