hund030 / AiGallery

0 stars 0 forks source link

Issue report #25

Closed ai-apps-bot closed 2 days ago

ai-apps-bot commented 1 week ago

AI Gallery Standard Validation: FAILED

The Definition of Done of the AI-Gallery repo can be found Here.

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Features is missing in README.md. - Error: ## Getting Started is missing in README.md. - Error: ## Guidance is missing in README.md. - Error: ## Resources is missing in README.md.
:heavy_check_mark: LICENSE File.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:x: .github/CODE_OF_CONDUCT.md File. [How to fix?] - Error: .github/CODE_OF_CONDUCT.md file is missing.
:x: CONTRIBUTING.md File. [How to fix?] - Error: CONTRIBUTING.md file is missing.
:x: .github/ISSUE_TEMPLATE.md File. [How to fix?] - Error: .github/ISSUE_TEMPLATE.md file is missing.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:heavy_check_mark: .github/workflows/azure-dev.yml File.
:heavy_check_mark: .github/workflows/pr-gate.yml File.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: ERROR: no project exists; to create a new project, run `azd init`
:x: azd down. [How to fix?] Error: ERROR: no project exists; to create a new project, run `azd init`

Security Requirements:

:heavy_check_mark: microsoft/security-devops-action is integrated to the CI/CD pipeline.
:warning: Security scan. [How to fix?] - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required.
ai-apps-bot commented 2 days ago

Failed/Total: 16/16 Failed Repos: agent-openai-python-prompty-langchain-pinecone agent-openai-python-prompty agent-python-openai-prompty-langchain azure-openai-assistant-javascript azure-search-openai-demo-csharp azure-search-openai-demo-java azure-search-openai-demo azure-search-openai-javascript contoso-chat-csharp-prompty contoso-chat openai-chat-app-quickstart openai-plugin-fastapi rag-postgres-openai-python serverless-chat-langchainjs summarization-openai-csharp-prompty summarization-openai-python-promptflow

AI Gallery Standard Validation: FAILED for Azure-Samples/agent-openai-python-prompty-langchain-pinecone

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Guidance is missing in README.md.
:heavy_check_mark: LICENSE File.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: azd-templates is missing in topics. - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:heavy_check_mark: .github/workflows/azure-dev.yml File.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Packaging services (azd package) Packaging service chat Packaging service chat (Copying deployment package) (✓) Done: Packaging service chat Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time. Initialize bicep provider Retrieving subscriptions... Reading subscription and location from environment... Subscription: Visual Studio Enterprise Subscription (6e41a27a-b56f-423c-b5d9-bb0b325733eb) Location: East US 2 Comparing deployment state Comparing deployment state Creating/Updating resources You can view detailed progress in the Azure Portal: https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2F6e41a27a-b56f-423c-b5d9-bb0b325733eb%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdev-07030857-1719997052 (✓) Done: Resource group: rg-dev-07030857 (✓) Done: Log Analytics workspace: log-7f4pci65jjxco (✓) Done: Key Vault: kv-7f4pci65jjxco (✓) Done: Storage account: st7f4pci65jjxco (✓) Done: Container Registry: cr7f4pci65jjxco ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: deploying to subscription: Deployment Error Details: DeploymentActive: Unable to edit or replace deployment 'user-role-secrets-reader': previous deployment from '7/3/2024 8:57:35 AM' is still active (expiration time is '7/10/2024 8:57:35 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. DeploymentActive: Unable to edit or replace deployment 'ai': previous deployment from '7/3/2024 8:57:35 AM' is still active (expiration time is '7/10/2024 8:57:34 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. DeploymentActive: Unable to edit or replace deployment 'user-role-data-scientist': previous deployment from '7/3/2024 8:57:35 AM' is still active (expiration time is '7/10/2024 8:57:34 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. DeploymentActive: Unable to edit or replace deployment 'user-acr-role-push': previous deployment from '7/3/2024 8:57:35 AM' is still active (expiration time is '7/10/2024 8:57:34 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. DeploymentActive: Unable to edit or replace deployment 'user-acr-role-pull': previous deployment from '7/3/2024 8:57:35 AM' is still active (expiration time is '7/10/2024 8:57:34 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. TraceID: 00000000000000000000000000000000
:heavy_check_mark: azd down.

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?] - Error: microsoft/security-devops-action is missing in .github/workflows/azure-dev.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/azure-dev.yml.
:warning: Security scan. [How to fix?] - error: AZR-000202 - By default, storage accounts accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. - error: AZR-000198 - Blob containers in Azure Storage Accounts can be configured for private or anonymous public access. By default, containers are private and only accessible with a credential or access token. When a container is configured with an access type other than private, anonymous access is permitted. Anonymous access to blobs or containers can be restricted by setting allowBlobPublicAccess to false. This enhanced security setting for a storage account overrides the individual settings for blob containers. When you disallow public access for a storage account, blobs are no longer accessible anonymously. - error: AZR-000355 - By default, Key Vault accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. If any of the following options are enabled you must also enable Allow trusted Microsoft services to bypass this firewall: - enabledForDeployment - Azure Virtual Machines for deployment. - enabledForDiskEncryption - Azure Disk Encryption for volume encryption. - enabledForTemplateDeployment - Azure Resource Manager for template deployment. - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: AZR-000291 - Every request to an Azure App Configuration resource must be authenticated. App Configuration supports authenticating requests using either Entra ID (previously Azure AD) identities or access keys. Using Entra ID identities: - Centralizes identity management and auditing. - Allows granting of permissions using role-based access control (RBAC). - Provides support for advanced security features such as conditional access and multi-factor authentication (MFA) when applicable. To require clients to use Entra ID to authenticate requests, you can disable the usage of access keys for an Azure App Configuration resource. When you disable access key authentication for an Azure App Configuration resource, any existing access keys for that resource are deleted. Any subsequent requests to the resource using the previously existing access keys will be rejected. Only requests that are authenticated using Entra ID will succeed. - error: AZR-000053 - API Management must authenticate to access Azure resources such as Key Vault. Use Key Vault to store certificates and secrets used within API Management. - error: AZR-000055 - API Management provides support for weak or deprecated ciphers. These older versions are provided for compatibility with clients and backends but are not consider secure. These many of these ciphers are enabled by default and need to be set to 'False'. The following ciphers are considered weak or deprecated: - TripleDes168 - TLS_RSA_WITH_AES_128_CBC_SHA - TLS_RSA_WITH_AES_256_CBC_SHA - TLS_RSA_WITH_AES_128_CBC_SHA256 - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA - TLS_RSA_WITH_AES_256_CBC_SHA256 - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA - TLS_RSA_WITH_AES_128_GCM_SHA256. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised. - error: AZR-000361 - Using managed identities have the following benefits: - Your app connects to resources with the managed identity. You don't need to manage credentials in your container app. - You can use role-based access control to grant specific permissions to a managed identity. - System-assigned identities are automatically created and managed. They're deleted when your container app is deleted. - You can add and delete user-assigned identities and assign them to multiple resources. They're independent of your container app's life cycle. - You can use managed identity to authenticate with a private Azure Container Registry without a username and password to pull containers for your Container App. - You can use managed identity to create connections for Dapr-enabled applications via Dapr components. - error: TA-000023 - To ensure that only applications from allowed networks, machines, or subnets can access your cluster, restrict access to your Kubernetes Service Management API server. It is recommended to limit access to authorized IP ranges to ensure that only applications from allowed networks can access the cluster. - error: AZR-000022 - To capture security-based audit logs from AKS clusters, the following diagnostic log categories should be enabled: - kube-audit or kube-audit-admin, or both. - kube-audit - Contains all audit log data for every audit event, including get, list, create, update, delete, patch, and post. - kube-audit-admin - Is a subset of the kube-audit log category. kube-audit-admin reduces the number of logs significantly by excluding the get and list audit events from the log. - guard - Contains logs for Azure Active Directory (AAD) authorization integration. For managed Azure AD, this includes token in and user info out. For Azure RBAC, this includes access reviews in and out. - error: AZR-000028 - AKS clusters support integration with Azure Policy using an Open Policy Agent (OPA). Azure Policy integration is provided by an optional add-on that can be enabled on AKS clusters. Once enabled and Azure policies assigned, AKS clusters will enforce the configured constraints. Examples of policies include: - Enforce HTTPS ingress in Kubernetes cluster. - Do not allow privileged containers in Kubernetes cluster. - Ensure container CPU and memory resource limits do not exceed the specified limits in Kubernetes cluster. - error: AZR-000029 - AKS-managed integration provides an easy way to use Azure AD authorization for AKS. Previous Azure AD integration with AKS required app registration and management within Azure AD. - error: AZR-000030 - In Kubernetes, the API server is the control plane of the cluster. Access to the API server is required by various cluster functions as well as all administrator activities. All activities performed against the cluster require authorization. To improve cluster security, the API server can be restricted to a limited set of IP address ranges. Restricting authorized IP addresses for the API server has the following limitations: - Requires AKS clusters configured with a Standard Load Balancer SKU. - This feature is not compatible with clusters that use Public IP per Node. - This feature is not compatible with AKS private clusters. When configuring this feature, you must specify the IP address ranges that will be authorized. To allow only the outbound public IP of the Standard SKU load balancer, use 0.0.0.0/32. You should add these ranges to the allow list: - Include output IP addresses for cluster nodes - Any range where administration will connect to the API server, including CI/CD systems, monitoring, and management systems. - error: AZR-000032 - Azure Kubernetes Service (AKS) supports Role-based Access Control (RBAC). RBAC is supported using Kubernetes RBAC and optionally Azure RBAC. - Using Kubernetes RBAC, you can grant users, groups, and service accounts access to cluster resources. - Additionally AKS supports granting Azure AD identities access to cluster resources using Azure RBAC. Using authorization provided by Azure RBAC simplifies and centralizes authorization of Azure AD principals. Access to Kubernetes resource can be managed using Azure Resource Manager (ARM). When Azure RBAC is enabled: - Azure AD principals will be validated exclusively by Azure RBAC. - Kubernetes users and service accounts are exclusively validated by Kubernetes RBAC. - error: AZR-000033 - AKS clusters may need to store and retrieve secrets, keys, and certificates. The Secrets Store CSI Driver provides cluster support to integrate with Key Vault. When enabled and configured secrets, keys, and certificates can be securely accessed from a pod. The Secrets Store CSI Driver can automatically refresh secrets and keys periodically from Key Vault. To enable this feature, enable Secrets Store CSI Driver autorotation. Avoid storing secrets to access Azure resources. Use a Managed Identity when possible instead of cryptographic keys or a regular service principal. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - warning: AZR-000388 - Azure RBAC is the recommended authorization system for the Azure Key Vault data plane. Azure RBAC allows users to manage key, secrets, and certificates permissions. It provides one place to manage all permissions across all Key Vaults. Azure RBAC for Key Vault also allows users to have separate permissions on individual keys, secrets, and certificates. The Azure RBAC permission model is not enabled by default. - warning: AZR-000406 - Disabling public network access improves security by ensuring that the Machine Learning Workspaces aren't exposed on the public internet. You can control exposure of your workspaces by creating private endpoints instead. By default, a public endpoint is enabled for Machine Learning workspaces. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help control exposure of a workspace to data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - warning: AZR-000407 - Manange access to Azure ML workspace and associated resources, Azure Container Registry, KeyVault, Storage, and App Insights using user-assigned managed identity. By default, system-assigned managed identity is used by Azure ML workspace to access the associated resources. User-assigned managed identity allows you to create the identity as an Azure resource and maintain the life cycle of that identity. - warning: AZR-000406 - Disabling public network access improves security by ensuring that the Machine Learning Workspaces aren't exposed on the public internet. You can control exposure of your workspaces by creating private endpoints instead. By default, a public endpoint is enabled for Machine Learning workspaces. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help control exposure of a workspace to data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - warning: AZR-000407 - Manange access to Azure ML workspace and associated resources, Azure Container Registry, KeyVault, Storage, and App Insights using user-assigned managed identity. By default, system-assigned managed identity is used by Azure ML workspace to access the associated resources. User-assigned managed identity allows you to create the identity as an Azure resource and maintain the life cycle of that identity. - warning: AZR-000031 - AKS clusters support Role-based Access Control (RBAC) authorization. RBAC allows users, groups, and service accounts to be granted access to resources on an as needed basis. Actions performed by each identity can be logged for auditing with Kubernetes audit policies. When a cluster is deployed, local accounts are enabled by default even when RBAC is enabled. These local accounts such as clusterAdmin and clusterUser are shared accounts that are not tied to an identity. If local account credentials are used, Kubernetes auditing logs the local account instead of named accounts. Who performed an action cannot be determined from the audit logs, creating an audit log gap for privileged actions. In an AKS cluster with local account disabled administrator will be unable to get the clusterAdmin credential. For example, using az aks get-credentials -g '' -n '' --admin will fail. - warning: AZR-000370 - To collect and provide data plane protections of Microsoft Defender for Containers some additional daemon set and deployments needs to be deployed to the AKS clusters. These components are installed when the Defender profile is enabled on the cluster. The Defender profile deployed to each node provides the runtime protections and collects signals from nodes.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/agent-openai-python-prompty

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Getting Started is missing in README.md. - Error: ## Guidance is missing in README.md.
:heavy_check_mark: LICENSE File.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: azd-templates is missing in topics. - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:heavy_check_mark: .github/workflows/azure-dev.yml File.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Packaging services (azd package) Packaging service api Packaging service api (Tagging container image) (✓) Done: Packaging service api - Image Hash: sha256:d1b737e25cb4ea53ab937bc34b032f7ffdcdb8a08004db6d2649c67ebf38e10a - Target Image: creativeagent/api-dev-07030856:azd-deploy-1719997107 Packaging service web Packaging service web (Tagging container image) (✓) Done: Packaging service web - Image Hash: sha256:9b6043ae388d976a3c14596a61a5dd407f4a5b156766f8e2dc7f056d8a5a9197 - Target Image: creativeagent/web-dev-07030856:azd-deploy-1719997137 Created and switched to workspace "azure"! You're now on a new, empty workspace. Workspaces isolate their state, so if you run "terraform plan" Terraform will not see any existing state for this configuration. Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time. Retrieving subscriptions... Reading subscription and location from environment... Subscription: Visual Studio Enterprise Subscription (6e41a27a-b56f-423c-b5d9-bb0b325733eb) Location: East US 2 Locating plan file... Generating terraform backend config file... Initializing the backend... ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: terraform init failed: , err: failed running terraform init: (exit code: 1) TraceID: 00000000000000000000000000000000
:x: azd down. [How to fix?] Error: Deleting all resources and deployed code on Azure (azd down) Local application code is not deleted when running 'azd down'. Locating parameters file... ERROR: deleting infrastructure: error deleting Azure resources: load terraform template output failed: reading deployment output failed: , err:failed running terraform output: ╷ │ Error: Backend initialization required, please run "terraform init" │  │ Reason: Initial configuration of the requested backend "azurerm" │  │ The "backend" is the interface that Terraform uses to store state, │ perform operations, etc. If this message is showing up, it means that the │ Terraform configuration you're using is using a custom configuration for │ the Terraform backend. │  │ Changes to backend configurations require reinitialization. This allows │ Terraform to set up the new configuration, copy existing state, etc. Please │ run │ "terraform init" with either the "-reconfigure" or "-migrate-state" flags │ to │ use the current configuration. │  │ If the change reason above is incorrect, please verify your configuration │ hasn't changed and try again. At this point, no changes to your existing │ configuration or state have been made. ╵ (exit code: 1, stdout: , stderr: ╷ │ Error: Backend initialization required, please run "terraform init" │  │ Reason: Initial configuration of the requested backend "azurerm" │  │ The "backend" is the interface that Terraform uses to store state, │ perform operations, etc. If this message is showing up, it means that the │ Terraform configuration you're using is using a custom configuration for │ the Terraform backend. │  │ Changes to backend configurations require reinitialization. This allows │ Terraform to set up the new configuration, copy existing state, etc. Please │ run │ "terraform init" with either the "-reconfigure" or "-migrate-state" flags │ to │ use the current configuration. │  │ If the change reason above is incorrect, please verify your configuration │ hasn't changed and try again. At this point, no changes to your existing │ configuration or state have been made. ╵ ) TraceID: 00000000000000000000000000000000

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?] - Error: microsoft/security-devops-action is missing in .github/workflows/azure-dev.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/azure-dev.yml. - Error: microsoft/security-devops-action is missing in .github/workflows/evaluate.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/evaluate.yml.
:heavy_check_mark: Security scan.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/agent-python-openai-prompty-langchain

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Guidance is missing in README.md.
:heavy_check_mark: LICENSE File.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: azd-templates is missing in topics. - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:heavy_check_mark: .github/workflows/azure-dev.yml File.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Packaging services (azd package) Packaging service chat Packaging service chat (Copying deployment package) (✓) Done: Packaging service chat Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time. Initialize bicep provider Retrieving subscriptions... Reading subscription and location from environment... Subscription: Visual Studio Enterprise Subscription (6e41a27a-b56f-423c-b5d9-bb0b325733eb) Location: East US 2 Creating a deployment plan Comparing deployment state Creating/Updating resources You can view detailed progress in the Azure Portal: https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2F6e41a27a-b56f-423c-b5d9-bb0b325733eb%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdev-07030857-1719997061 (✓) Done: Resource group: rg-dev-07030857 (✓) Done: Log Analytics workspace: log-7f4pci65jjxco (✓) Done: Key Vault: kv-7f4pci65jjxco (✓) Done: Storage account: st7f4pci65jjxco (✓) Done: Container Registry: cr7f4pci65jjxco ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: deploying to subscription: Deployment Error Details: UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. DeploymentActive: Unable to edit or replace deployment 'ai': previous deployment from '7/3/2024 8:57:36 AM' is still active (expiration time is '7/10/2024 8:57:34 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. TraceID: 00000000000000000000000000000000
:x: azd down. [How to fix?] Error: Deleting all resources and deployed code on Azure (azd down) Local application code is not deleted when running 'azd down'. Initialize bicep provider Deleting your resources can take some time. Deleting resource group: rg-dev-07030857 (✓) Done: Deleting resource group: rg-dev-07030857 Purging Key Vault: kv-7f4pci65jjxco (x) Failed: Purging Key Vault: kv-7f4pci65jjxco ERROR: deleting infrastructure: error deleting Azure resources: purging resources: failed to purge Key Vault: purging key vault kv-7f4pci65jjxco: starting purging key vault: POST https://management.azure.com/subscriptions/6e41a27a-b56f-423c-b5d9-bb0b325733eb/providers/Microsoft.KeyVault/locations/eastus2/deletedVaults/kv-7f4pci65jjxco/purge -------------------------------------------------------------------------------- RESPONSE 404: 404 Not Found ERROR CODE: DeletedVaultNotFound -------------------------------------------------------------------------------- { "error": { "code": "DeletedVaultNotFound", "message": "The specified deleted vault 'kv-7f4pci65jjxco' does not exist. Ensure that the vault was indeed deleted and that it is in recoverable state. If soft delete was not enabled then the vault is permanently deleted. Follow this link for more information: https://go.microsoft.com/fwlink/?linkid=2149745" } } -------------------------------------------------------------------------------- TraceID: 00000000000000000000000000000000

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?] - Error: microsoft/security-devops-action is missing in .github/workflows/azure-dev.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/azure-dev.yml.
:warning: Security scan. [How to fix?] - error: AZR-000202 - By default, storage accounts accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. - error: AZR-000198 - Blob containers in Azure Storage Accounts can be configured for private or anonymous public access. By default, containers are private and only accessible with a credential or access token. When a container is configured with an access type other than private, anonymous access is permitted. Anonymous access to blobs or containers can be restricted by setting allowBlobPublicAccess to false. This enhanced security setting for a storage account overrides the individual settings for blob containers. When you disallow public access for a storage account, blobs are no longer accessible anonymously. - error: AZR-000355 - By default, Key Vault accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. If any of the following options are enabled you must also enable Allow trusted Microsoft services to bypass this firewall: - enabledForDeployment - Azure Virtual Machines for deployment. - enabledForDiskEncryption - Azure Disk Encryption for volume encryption. - enabledForTemplateDeployment - Azure Resource Manager for template deployment. - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: AZR-000291 - Every request to an Azure App Configuration resource must be authenticated. App Configuration supports authenticating requests using either Entra ID (previously Azure AD) identities or access keys. Using Entra ID identities: - Centralizes identity management and auditing. - Allows granting of permissions using role-based access control (RBAC). - Provides support for advanced security features such as conditional access and multi-factor authentication (MFA) when applicable. To require clients to use Entra ID to authenticate requests, you can disable the usage of access keys for an Azure App Configuration resource. When you disable access key authentication for an Azure App Configuration resource, any existing access keys for that resource are deleted. Any subsequent requests to the resource using the previously existing access keys will be rejected. Only requests that are authenticated using Entra ID will succeed. - error: AZR-000053 - API Management must authenticate to access Azure resources such as Key Vault. Use Key Vault to store certificates and secrets used within API Management. - error: AZR-000055 - API Management provides support for weak or deprecated ciphers. These older versions are provided for compatibility with clients and backends but are not consider secure. These many of these ciphers are enabled by default and need to be set to 'False'. The following ciphers are considered weak or deprecated: - TripleDes168 - TLS_RSA_WITH_AES_128_CBC_SHA - TLS_RSA_WITH_AES_256_CBC_SHA - TLS_RSA_WITH_AES_128_CBC_SHA256 - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA - TLS_RSA_WITH_AES_256_CBC_SHA256 - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA - TLS_RSA_WITH_AES_128_GCM_SHA256. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised. - error: AZR-000361 - Using managed identities have the following benefits: - Your app connects to resources with the managed identity. You don't need to manage credentials in your container app. - You can use role-based access control to grant specific permissions to a managed identity. - System-assigned identities are automatically created and managed. They're deleted when your container app is deleted. - You can add and delete user-assigned identities and assign them to multiple resources. They're independent of your container app's life cycle. - You can use managed identity to authenticate with a private Azure Container Registry without a username and password to pull containers for your Container App. - You can use managed identity to create connections for Dapr-enabled applications via Dapr components. - error: TA-000023 - To ensure that only applications from allowed networks, machines, or subnets can access your cluster, restrict access to your Kubernetes Service Management API server. It is recommended to limit access to authorized IP ranges to ensure that only applications from allowed networks can access the cluster. - error: AZR-000022 - To capture security-based audit logs from AKS clusters, the following diagnostic log categories should be enabled: - kube-audit or kube-audit-admin, or both. - kube-audit - Contains all audit log data for every audit event, including get, list, create, update, delete, patch, and post. - kube-audit-admin - Is a subset of the kube-audit log category. kube-audit-admin reduces the number of logs significantly by excluding the get and list audit events from the log. - guard - Contains logs for Azure Active Directory (AAD) authorization integration. For managed Azure AD, this includes token in and user info out. For Azure RBAC, this includes access reviews in and out. - error: AZR-000028 - AKS clusters support integration with Azure Policy using an Open Policy Agent (OPA). Azure Policy integration is provided by an optional add-on that can be enabled on AKS clusters. Once enabled and Azure policies assigned, AKS clusters will enforce the configured constraints. Examples of policies include: - Enforce HTTPS ingress in Kubernetes cluster. - Do not allow privileged containers in Kubernetes cluster. - Ensure container CPU and memory resource limits do not exceed the specified limits in Kubernetes cluster. - error: AZR-000029 - AKS-managed integration provides an easy way to use Azure AD authorization for AKS. Previous Azure AD integration with AKS required app registration and management within Azure AD. - error: AZR-000030 - In Kubernetes, the API server is the control plane of the cluster. Access to the API server is required by various cluster functions as well as all administrator activities. All activities performed against the cluster require authorization. To improve cluster security, the API server can be restricted to a limited set of IP address ranges. Restricting authorized IP addresses for the API server has the following limitations: - Requires AKS clusters configured with a Standard Load Balancer SKU. - This feature is not compatible with clusters that use Public IP per Node. - This feature is not compatible with AKS private clusters. When configuring this feature, you must specify the IP address ranges that will be authorized. To allow only the outbound public IP of the Standard SKU load balancer, use 0.0.0.0/32. You should add these ranges to the allow list: - Include output IP addresses for cluster nodes - Any range where administration will connect to the API server, including CI/CD systems, monitoring, and management systems. - error: AZR-000032 - Azure Kubernetes Service (AKS) supports Role-based Access Control (RBAC). RBAC is supported using Kubernetes RBAC and optionally Azure RBAC. - Using Kubernetes RBAC, you can grant users, groups, and service accounts access to cluster resources. - Additionally AKS supports granting Azure AD identities access to cluster resources using Azure RBAC. Using authorization provided by Azure RBAC simplifies and centralizes authorization of Azure AD principals. Access to Kubernetes resource can be managed using Azure Resource Manager (ARM). When Azure RBAC is enabled: - Azure AD principals will be validated exclusively by Azure RBAC. - Kubernetes users and service accounts are exclusively validated by Kubernetes RBAC. - error: AZR-000033 - AKS clusters may need to store and retrieve secrets, keys, and certificates. The Secrets Store CSI Driver provides cluster support to integrate with Key Vault. When enabled and configured secrets, keys, and certificates can be securely accessed from a pod. The Secrets Store CSI Driver can automatically refresh secrets and keys periodically from Key Vault. To enable this feature, enable Secrets Store CSI Driver autorotation. Avoid storing secrets to access Azure resources. Use a Managed Identity when possible instead of cryptographic keys or a regular service principal. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - warning: AZR-000388 - Azure RBAC is the recommended authorization system for the Azure Key Vault data plane. Azure RBAC allows users to manage key, secrets, and certificates permissions. It provides one place to manage all permissions across all Key Vaults. Azure RBAC for Key Vault also allows users to have separate permissions on individual keys, secrets, and certificates. The Azure RBAC permission model is not enabled by default. - warning: AZR-000406 - Disabling public network access improves security by ensuring that the Machine Learning Workspaces aren't exposed on the public internet. You can control exposure of your workspaces by creating private endpoints instead. By default, a public endpoint is enabled for Machine Learning workspaces. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help control exposure of a workspace to data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - warning: AZR-000407 - Manange access to Azure ML workspace and associated resources, Azure Container Registry, KeyVault, Storage, and App Insights using user-assigned managed identity. By default, system-assigned managed identity is used by Azure ML workspace to access the associated resources. User-assigned managed identity allows you to create the identity as an Azure resource and maintain the life cycle of that identity. - warning: AZR-000406 - Disabling public network access improves security by ensuring that the Machine Learning Workspaces aren't exposed on the public internet. You can control exposure of your workspaces by creating private endpoints instead. By default, a public endpoint is enabled for Machine Learning workspaces. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help control exposure of a workspace to data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - warning: AZR-000407 - Manange access to Azure ML workspace and associated resources, Azure Container Registry, KeyVault, Storage, and App Insights using user-assigned managed identity. By default, system-assigned managed identity is used by Azure ML workspace to access the associated resources. User-assigned managed identity allows you to create the identity as an Azure resource and maintain the life cycle of that identity. - warning: AZR-000031 - AKS clusters support Role-based Access Control (RBAC) authorization. RBAC allows users, groups, and service accounts to be granted access to resources on an as needed basis. Actions performed by each identity can be logged for auditing with Kubernetes audit policies. When a cluster is deployed, local accounts are enabled by default even when RBAC is enabled. These local accounts such as clusterAdmin and clusterUser are shared accounts that are not tied to an identity. If local account credentials are used, Kubernetes auditing logs the local account instead of named accounts. Who performed an action cannot be determined from the audit logs, creating an audit log gap for privileged actions. In an AKS cluster with local account disabled administrator will be unable to get the clusterAdmin credential. For example, using az aks get-credentials -g '' -n '' --admin will fail. - warning: AZR-000370 - To collect and provide data plane protections of Microsoft Defender for Containers some additional daemon set and deployments needs to be deployed to the AKS clusters. These components are installed when the Defender profile is enabled on the cluster. The Defender profile deployed to each node provides the runtime protections and collects signals from nodes.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/azure-openai-assistant-javascript

Repository Management:

:heavy_check_mark: README.md File.
:x: LICENSE File. [How to fix?] - Error: LICENSE file is missing.
:heavy_check_mark: SECURITY.md File.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: azd-templates is missing in topics. - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:x: .github/workflows/azure-dev.yml File. [How to fix?] - Error: .github/workflows/azure-dev.yml file is missing.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Packaging services (azd package) Packaging service api Packaging service api (Installing NPM dependencies) Packaging service api (Running NPM package script) Packaging service api (Copying deployment package) Packaging service api (Compressing deployment artifacts) (✓) Done: Packaging service api - Package Output: /tmp/azure-openai-assistant-javascript@1.0.0-api-azddeploy-1719997035.zip Packaging service webapp Packaging service webapp (Installing NPM dependencies) Packaging service webapp (Running NPM package script) Packaging service webapp (Copying deployment package) (✓) Done: Packaging service webapp - Package Output: ../dist Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time. Initialize bicep provider Retrieving subscriptions... Reading subscription and location from environment... Subscription: Visual Studio Enterprise Subscription (6e41a27a-b56f-423c-b5d9-bb0b325733eb) Location: East US 2 Comparing deployment state Comparing deployment state Creating/Updating resources You can view detailed progress in the Azure Portal: https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2F6e41a27a-b56f-423c-b5d9-bb0b325733eb%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdev-07030856-1719997055 (✓) Done: Resource group: rg-dev-07030856 (✓) Done: Static Web App: webapp (✓) Done: App Service plan: plan-fq7hgyla7b2aw (✓) Done: Storage account: stfq7hgyla7b2aw (✓) Done: Azure OpenAI: cog-fq7hgyla7b2aw (✓) Done: Function App: func-api-fq7hgyla7b2aw ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: deploying to subscription: Deployment Error Details: UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. TraceID: 00000000000000000000000000000000
:heavy_check_mark: azd down.

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?]
:warning: Security scan. [How to fix?] - error: AZR-000202 - By default, storage accounts accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. - error: AZR-000198 - Blob containers in Azure Storage Accounts can be configured for private or anonymous public access. By default, containers are private and only accessible with a credential or access token. When a container is configured with an access type other than private, anonymous access is permitted. Anonymous access to blobs or containers can be restricted by setting allowBlobPublicAccess to false. This enhanced security setting for a storage account overrides the individual settings for blob containers. When you disallow public access for a storage account, blobs are no longer accessible anonymously. - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/azure-search-openai-demo-csharp

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Guidance is missing in README.md.
:heavy_check_mark: LICENSE File.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:heavy_check_mark: .github/workflows/azure-dev.yml File.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider ERROR: prompting for value: no default response for prompt 'Enter a value for the 'openAIApiKey' infrastructure parameter:'
:x: azd down. [How to fix?] Error: Deleting all resources and deployed code on Azure (azd down) Local application code is not deleted when running 'azd down'. Initialize bicep provider Enter a value for the 'openAIApiKey' infrastructure parameter: Deleting your resources can take some time. Deleting resource group: rg-dev-07030857 (✓) Done: Deleting resource group: rg-dev-07030857 Purging Key Vault: kv-7f4pci65jjxco (x) Failed: Purging Key Vault: kv-7f4pci65jjxco ERROR: deleting infrastructure: error deleting Azure resources: purging resources: failed to purge Key Vault: purging key vault kv-7f4pci65jjxco: starting purging key vault: POST https://management.azure.com/subscriptions/6e41a27a-b56f-423c-b5d9-bb0b325733eb/providers/Microsoft.KeyVault/locations/eastus2/deletedVaults/kv-7f4pci65jjxco/purge -------------------------------------------------------------------------------- RESPONSE 404: 404 Not Found ERROR CODE: DeletedVaultNotFound -------------------------------------------------------------------------------- { "error": { "code": "DeletedVaultNotFound", "message": "The specified deleted vault 'kv-7f4pci65jjxco' does not exist. Ensure that the vault was indeed deleted and that it is in recoverable state. If soft delete was not enabled then the vault is permanently deleted. Follow this link for more information: https://go.microsoft.com/fwlink/?linkid=2149745" } } -------------------------------------------------------------------------------- TraceID: 00000000000000000000000000000000

Security Requirements:

:heavy_check_mark: microsoft/security-devops-action is integrated to the CI/CD pipeline.
:warning: Security scan. [How to fix?] - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised. - error: AZR-000202 - By default, storage accounts accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. - error: AZR-000198 - Blob containers in Azure Storage Accounts can be configured for private or anonymous public access. By default, containers are private and only accessible with a credential or access token. When a container is configured with an access type other than private, anonymous access is permitted. Anonymous access to blobs or containers can be restricted by setting allowBlobPublicAccess to false. This enhanced security setting for a storage account overrides the individual settings for blob containers. When you disallow public access for a storage account, blobs are no longer accessible anonymously. - error: AZR-000355 - By default, Key Vault accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. If any of the following options are enabled you must also enable Allow trusted Microsoft services to bypass this firewall: - enabledForDeployment - Azure Virtual Machines for deployment. - enabledForDiskEncryption - Azure Disk Encryption for volume encryption. - enabledForTemplateDeployment - Azure Resource Manager for template deployment. - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: AZR-000361 - Using managed identities have the following benefits: - Your app connects to resources with the managed identity. You don't need to manage credentials in your container app. - You can use role-based access control to grant specific permissions to a managed identity. - System-assigned identities are automatically created and managed. They're deleted when your container app is deleted. - You can add and delete user-assigned identities and assign them to multiple resources. They're independent of your container app's life cycle. - You can use managed identity to authenticate with a private Azure Container Registry without a username and password to pull containers for your Container App. - You can use managed identity to create connections for Dapr-enabled applications via Dapr components. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - warning: AZR-000388 - Azure RBAC is the recommended authorization system for the Azure Key Vault data plane. Azure RBAC allows users to manage key, secrets, and certificates permissions. It provides one place to manage all permissions across all Key Vaults. Azure RBAC for Key Vault also allows users to have separate permissions on individual keys, secrets, and certificates. The Azure RBAC permission model is not enabled by default.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/azure-search-openai-demo-java

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Features is missing in README.md. - Error: ## Getting Started is missing in README.md. - Error: ## Guidance is missing in README.md. - Error: ## Resources is missing in README.md.
:heavy_check_mark: LICENSE File.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: azd-templates is missing in topics. - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:x: .github/workflows/azure-dev.yml File. [How to fix?] - Error: .github/workflows/azure-dev.yml file is missing.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:x: azure.yaml File. [How to fix?] - Error: azure.yaml file is missing.
:x: infra Folder. [How to fix?] - Error: infra folder is missing.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: ERROR: no project exists; to create a new project, run `azd init`
:x: azd down. [How to fix?] Error: ERROR: no project exists; to create a new project, run `azd init`

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?] - Error: microsoft/security-devops-action is missing in .github/workflows/stale-bot.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/stale-bot.yml.
:warning: Security scan. [How to fix?] - error: AZR-000202 - By default, storage accounts accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. - error: AZR-000198 - Blob containers in Azure Storage Accounts can be configured for private or anonymous public access. By default, containers are private and only accessible with a credential or access token. When a container is configured with an access type other than private, anonymous access is permitted. Anonymous access to blobs or containers can be restricted by setting allowBlobPublicAccess to false. This enhanced security setting for a storage account overrides the individual settings for blob containers. When you disallow public access for a storage account, blobs are no longer accessible anonymously. - error: AZR-000178 - To publish or consume messages from Service Bus cryptographic keys, or Azure AD identities can be used. Cryptographic keys include Shared Access Policy keys or Shared Access Signature (SAS) tokens. With Azure AD authentication, the identity is validated against Azure AD. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys or SAS tokens. - error: AZR-000355 - By default, Key Vault accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. If any of the following options are enabled you must also enable Allow trusted Microsoft services to bypass this firewall: - enabledForDeployment - Azure Virtual Machines for deployment. - enabledForDiskEncryption - Azure Disk Encryption for volume encryption. - enabledForTemplateDeployment - Azure Resource Manager for template deployment. - error: AZR-000099 - When delivering events you can use Managed Identities to authenticate event delivery. You can enable either system-assigned identity or user-assigned identity but not both. You can have at most two user-assigned identities assigned to a topic or domain. - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised. - error: AZR-000361 - Using managed identities have the following benefits: - Your app connects to resources with the managed identity. You don't need to manage credentials in your container app. - You can use role-based access control to grant specific permissions to a managed identity. - System-assigned identities are automatically created and managed. They're deleted when your container app is deleted. - You can add and delete user-assigned identities and assign them to multiple resources. They're independent of your container app's life cycle. - You can use managed identity to authenticate with a private Azure Container Registry without a username and password to pull containers for your Container App. - You can use managed identity to create connections for Dapr-enabled applications via Dapr components. - error: TA-000023 - To ensure that only applications from allowed networks, machines, or subnets can access your cluster, restrict access to your Kubernetes Service Management API server. It is recommended to limit access to authorized IP ranges to ensure that only applications from allowed networks can access the cluster. - error: AZR-000022 - To capture security-based audit logs from AKS clusters, the following diagnostic log categories should be enabled: - kube-audit or kube-audit-admin, or both. - kube-audit - Contains all audit log data for every audit event, including get, list, create, update, delete, patch, and post. - kube-audit-admin - Is a subset of the kube-audit log category. kube-audit-admin reduces the number of logs significantly by excluding the get and list audit events from the log. - guard - Contains logs for Azure Active Directory (AAD) authorization integration. For managed Azure AD, this includes token in and user info out. For Azure RBAC, this includes access reviews in and out. - error: AZR-000028 - AKS clusters support integration with Azure Policy using an Open Policy Agent (OPA). Azure Policy integration is provided by an optional add-on that can be enabled on AKS clusters. Once enabled and Azure policies assigned, AKS clusters will enforce the configured constraints. Examples of policies include: - Enforce HTTPS ingress in Kubernetes cluster. - Do not allow privileged containers in Kubernetes cluster. - Ensure container CPU and memory resource limits do not exceed the specified limits in Kubernetes cluster. - error: AZR-000029 - AKS-managed integration provides an easy way to use Azure AD authorization for AKS. Previous Azure AD integration with AKS required app registration and management within Azure AD. - error: AZR-000030 - In Kubernetes, the API server is the control plane of the cluster. Access to the API server is required by various cluster functions as well as all administrator activities. All activities performed against the cluster require authorization. To improve cluster security, the API server can be restricted to a limited set of IP address ranges. Restricting authorized IP addresses for the API server has the following limitations: - Requires AKS clusters configured with a Standard Load Balancer SKU. - This feature is not compatible with clusters that use Public IP per Node. - This feature is not compatible with AKS private clusters. When configuring this feature, you must specify the IP address ranges that will be authorized. To allow only the outbound public IP of the Standard SKU load balancer, use 0.0.0.0/32. You should add these ranges to the allow list: - Include output IP addresses for cluster nodes - Any range where administration will connect to the API server, including CI/CD systems, monitoring, and management systems. - error: AZR-000032 - Azure Kubernetes Service (AKS) supports Role-based Access Control (RBAC). RBAC is supported using Kubernetes RBAC and optionally Azure RBAC. - Using Kubernetes RBAC, you can grant users, groups, and service accounts access to cluster resources. - Additionally AKS supports granting Azure AD identities access to cluster resources using Azure RBAC. Using authorization provided by Azure RBAC simplifies and centralizes authorization of Azure AD principals. Access to Kubernetes resource can be managed using Azure Resource Manager (ARM). When Azure RBAC is enabled: - Azure AD principals will be validated exclusively by Azure RBAC. - Kubernetes users and service accounts are exclusively validated by Kubernetes RBAC. - error: AZR-000033 - AKS clusters may need to store and retrieve secrets, keys, and certificates. The Secrets Store CSI Driver provides cluster support to integrate with Key Vault. When enabled and configured secrets, keys, and certificates can be securely accessed from a pod. The Secrets Store CSI Driver can automatically refresh secrets and keys periodically from Key Vault. To enable this feature, enable Secrets Store CSI Driver autorotation. Avoid storing secrets to access Azure resources. Use a Managed Identity when possible instead of cryptographic keys or a regular service principal. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - warning: AZR-000388 - Azure RBAC is the recommended authorization system for the Azure Key Vault data plane. Azure RBAC allows users to manage key, secrets, and certificates permissions. It provides one place to manage all permissions across all Key Vaults. Azure RBAC for Key Vault also allows users to have separate permissions on individual keys, secrets, and certificates. The Azure RBAC permission model is not enabled by default. - warning: AZR-000031 - AKS clusters support Role-based Access Control (RBAC) authorization. RBAC allows users, groups, and service accounts to be granted access to resources on an as needed basis. Actions performed by each identity can be logged for auditing with Kubernetes audit policies. When a cluster is deployed, local accounts are enabled by default even when RBAC is enabled. These local accounts such as clusterAdmin and clusterUser are shared accounts that are not tied to an identity. If local account credentials are used, Kubernetes auditing logs the local account instead of named accounts. Who performed an action cannot be determined from the audit logs, creating an audit log gap for privileged actions. In an AKS cluster with local account disabled administrator will be unable to get the clusterAdmin credential. For example, using az aks get-credentials -g '' -n '' --admin will fail. - warning: AZR-000370 - To collect and provide data plane protections of Microsoft Defender for Containers some additional daemon set and deployments needs to be deployed to the AKS clusters. These components are installed when the Defender profile is enabled on the cluster. The Defender profile deployed to each node provides the runtime protections and collects signals from nodes.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/azure-search-openai-demo

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Getting Started is missing in README.md. - Error: ## Guidance is missing in README.md.
:heavy_check_mark: LICENSE File.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:heavy_check_mark: Topics on repo contains ['azd-templates', 'ai-azd-templates'].

Source code structure and conventions:

:heavy_check_mark: .github/workflows/azure-dev.yml File.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Retrieving locations... ERROR: prompting for value: prompting for location: no default response for prompt 'Enter a value for the 'documentIntelligenceResourceGroupLocation' infrastructure parameter:'
:x: azd down. [How to fix?] Error: Deleting all resources and deployed code on Azure (azd down) Local application code is not deleted when running 'azd down'. Initialize bicep provider Retrieving locations... Enter a value for the 'documentIntelligenceResourceGroupLocation' infrastructure parameter: ERROR: initializing provisioning manager: prompting for value: prompting for location: '' is not an allowed choice. allowed choices: 1. (Europe) West Europe (westeurope), 2. (US) East US (eastus), 3. (US) West US 2 (westus2)

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?] - Error: microsoft/security-devops-action is missing in .github/workflows/azure-dev.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/azure-dev.yml. - Error: microsoft/security-devops-action is missing in .github/workflows/lint-markdown.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/lint-markdown.yml. - Error: microsoft/security-devops-action is missing in .github/workflows/stale-bot.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/stale-bot.yml. - Error: microsoft/security-devops-action is missing in .github/workflows/validate-markdown.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/validate-markdown.yml. - Error: microsoft/security-devops-action is missing in .github/workflows/validate-markdown.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/validate-markdown.yml. - Error: microsoft/security-devops-action is missing in .github/workflows/validate-markdown.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/validate-markdown.yml.
:warning: Security scan. [How to fix?] - error: AZR-000202 - By default, storage accounts accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. - error: AZR-000198 - Blob containers in Azure Storage Accounts can be configured for private or anonymous public access. By default, containers are private and only accessible with a credential or access token. When a container is configured with an access type other than private, anonymous access is permitted. Anonymous access to blobs or containers can be restricted by setting allowBlobPublicAccess to false. This enhanced security setting for a storage account overrides the individual settings for blob containers. When you disallow public access for a storage account, blobs are no longer accessible anonymously. - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: TA-000019 - For enhanced authentication security, use a managed identity. On Azure, managed identities eliminate the need for developers to have to manage credentials by providing an identity for the Azure resource in Azure AD and using it to obtain Azure Active Directory (Azure AD) tokens.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/azure-search-openai-javascript

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Getting Started is missing in README.md. - Error: ## Guidance is missing in README.md.
:heavy_check_mark: LICENSE File.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:x: .github/workflows/azure-dev.yml File. [How to fix?] - Error: .github/workflows/azure-dev.yml file is missing.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Packaging services (azd package) Packaging service indexer Packaging service indexer (Tagging container image) (✓) Done: Packaging service indexer - Image Hash: sha256:7ba704bebe46e1da9aee350e2ef8daf568f32ad8701af410c7b8781296eb86b0 - Target Image: azure-search-openai-javascript/indexer-dev-07030856:azd-deploy-1719997081 Packaging service search Packaging service search (Tagging container image) (✓) Done: Packaging service search - Image Hash: sha256:12c5b5611b1ed4c66d08cd4727a2b4fb46e8f3517fe6d6e4fae9e0ee6a7f2101 - Target Image: azure-search-openai-javascript/search-dev-07030856:azd-deploy-1719997129 Packaging service webapp Packaging service webapp (Installing NPM dependencies) Packaging service webapp (Running NPM package script) Packaging service webapp (Copying deployment package) (✓) Done: Packaging service webapp - Package Output: dist Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time. Initialize bicep provider Retrieving subscriptions... Reading subscription and location from environment... Subscription: Visual Studio Enterprise Subscription (6e41a27a-b56f-423c-b5d9-bb0b325733eb) Location: East US 2 Creating a deployment plan Comparing deployment state Creating/Updating resources You can view detailed progress in the Azure Portal: https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2F6e41a27a-b56f-423c-b5d9-bb0b325733eb%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdev-07030856-1719997159 (✓) Done: Resource group: rg-dev-07030856 (✓) Done: Storage account: stfq7hgyla7b2aw (✓) Done: Log Analytics workspace: log-fq7hgyla7b2aw (✓) Done: Application Insights: appi-fq7hgyla7b2aw (✓) Done: Portal dashboard: dash-fq7hgyla7b2aw (x) Failed: Search service: gptkb-fq7hgyla7b2aw ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: deploying to subscription: Deployment Error Details: ResourceGroupBeingDeleted: The resource group 'rg-dev-07030856' is in deprovisioning state and cannot perform this operation. InvalidTemplateDeployment: The template deployment 'webapp' is not valid according to the validation procedure. The tracking id is '755460d4-a69c-4208-929f-345a23c45333'. See inner errors for details. ValidationForResourceFailed: Validation failed for a resource. Check 'Error.Details[0]' for more information. InvalidSkuName: Free SKU is invalid. Managed service identities are not allowed in this SKU. InvalidResourceLocation: The resource 'cog-fq7hgyla7b2aw' already exists in location 'swedencentral' in resource group 'rg-dev-07030856'. A resource with the same name cannot be created in location 'eastus2'. Please select a new resource name. TraceID: 00000000000000000000000000000000
:heavy_check_mark: azd down.

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?] - Error: microsoft/security-devops-action is missing in .github/workflows/stale-bot.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/stale-bot.yml. - Error: microsoft/security-devops-action is missing in .github/workflows/playwright.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/playwright.yml.
:warning: Security scan. [How to fix?] - error: AZR-000202 - By default, storage accounts accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. - error: AZR-000198 - Blob containers in Azure Storage Accounts can be configured for private or anonymous public access. By default, containers are private and only accessible with a credential or access token. When a container is configured with an access type other than private, anonymous access is permitted. Anonymous access to blobs or containers can be restricted by setting allowBlobPublicAccess to false. This enhanced security setting for a storage account overrides the individual settings for blob containers. When you disallow public access for a storage account, blobs are no longer accessible anonymously. - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: AZR-000361 - Using managed identities have the following benefits: - Your app connects to resources with the managed identity. You don't need to manage credentials in your container app. - You can use role-based access control to grant specific permissions to a managed identity. - System-assigned identities are automatically created and managed. They're deleted when your container app is deleted. - You can add and delete user-assigned identities and assign them to multiple resources. They're independent of your container app's life cycle. - You can use managed identity to authenticate with a private Azure Container Registry without a username and password to pull containers for your Container App. - You can use managed identity to create connections for Dapr-enabled applications via Dapr components. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/contoso-chat-csharp-prompty

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Features is missing in README.md. - Error: ## Getting Started is missing in README.md. - Error: ## Guidance is missing in README.md. - Error: ## Resources is missing in README.md.
:heavy_check_mark: LICENSE File.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: azd-templates is missing in topics. - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:heavy_check_mark: .github/workflows/azure-dev.yml File.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Retrieving locations... Retrieving locations... Packaging services (azd package) Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time. Initialize bicep provider Retrieving subscriptions... Reading subscription and location from environment... Subscription: Visual Studio Enterprise Subscription (6e41a27a-b56f-423c-b5d9-bb0b325733eb) Location: East US 2 Comparing deployment state Comparing deployment state Creating/Updating resources You can view detailed progress in the Azure Portal: https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2F6e41a27a-b56f-423c-b5d9-bb0b325733eb%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdev-07030857-1719997042 (✓) Done: Resource group: rg-dev-07030857 (✓) Done: Resource group: rg-dev-07030857 (x) Failed: Log Analytics workspace: dev-07030857-7f4pci65jjxco-loganalytics (✓) Done: Azure OpenAI: 7f4pci65jjxco-cog (✓) Done: Log Analytics workspace: log-7f4pci65jjxco (✓) Done: Log Analytics workspace: loganalytics (✓) Done: Container Registry: dev070308577f4pci65jjxcoregistry (✓) Done: Container Apps Environment: dev-07030857-7f4pci65jjxco-containerapps-env (✓) Done: Azure Cosmos DB: cosmos-contoso-7f4pci65jjxco (x) Failed: Search service: dev-07030857-7f4pci65jjxco-search-contoso ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: deploying to subscription: Deployment Error Details: ResourceGroupBeingDeleted: The resource group 'rg-dev-07030857' is in deprovisioning state and cannot perform this operation. DeploymentActive: Unable to edit or replace deployment 'container-apps': previous deployment from '7/3/2024 8:57:59 AM' is still active (expiration time is '7/10/2024 8:57:58 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. TraceID: 00000000000000000000000000000000
:heavy_check_mark: azd down.

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?] - Error: microsoft/security-devops-action is missing in .github/workflows/azure-dev.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/azure-dev.yml.
:warning: Security scan. [How to fix?] - error: AZR-000355 - By default, Key Vault accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. If any of the following options are enabled you must also enable Allow trusted Microsoft services to bypass this firewall: - enabledForDeployment - Azure Virtual Machines for deployment. - enabledForDiskEncryption - Azure Disk Encryption for volume encryption. - enabledForTemplateDeployment - Azure Resource Manager for template deployment. - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: AZR-000361 - Using managed identities have the following benefits: - Your app connects to resources with the managed identity. You don't need to manage credentials in your container app. - You can use role-based access control to grant specific permissions to a managed identity. - System-assigned identities are automatically created and managed. They're deleted when your container app is deleted. - You can add and delete user-assigned identities and assign them to multiple resources. They're independent of your container app's life cycle. - You can use managed identity to authenticate with a private Azure Container Registry without a username and password to pull containers for your Container App. - You can use managed identity to create connections for Dapr-enabled applications via Dapr components. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: AZR-000095 - Cosmos DB provides two authorization options for interacting with the database: - Azure Active Directory identity (Azure AD). Can be used to authorize account and resource management operations. - Keys and resource tokens. Can be used to authorize resource management and data operations. Resource management operations include management of databases, indexes, and containers. By default, keys are permitted to perform resource management operations. You can restrict these operations to Azure Resource Manager (ARM) calls only. - warning: AZR-000388 - Azure RBAC is the recommended authorization system for the Azure Key Vault data plane. Azure RBAC allows users to manage key, secrets, and certificates permissions. It provides one place to manage all permissions across all Key Vaults. Azure RBAC for Key Vault also allows users to have separate permissions on individual keys, secrets, and certificates. The Azure RBAC permission model is not enabled by default.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/contoso-chat

Repository Management:

:heavy_check_mark: README.md File.
:heavy_check_mark: LICENSE File.
:heavy_check_mark: SECURITY.md File.
:x: .github/CODE_OF_CONDUCT.md File. [How to fix?] - Error: .github/CODE_OF_CONDUCT.md file is missing.
:heavy_check_mark: CONTRIBUTING.md File.
:x: .github/ISSUE_TEMPLATE.md File. [How to fix?] - Error: .github/ISSUE_TEMPLATE.md file is missing.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: azd-templates is missing in topics. - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:heavy_check_mark: .github/workflows/azure-dev.yml File.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Note: Running custom 'up' workflow from azure.yaml Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time. Initialize bicep provider Retrieving subscriptions... Reading subscription and location from environment... Subscription: Visual Studio Enterprise Subscription (6e41a27a-b56f-423c-b5d9-bb0b325733eb) Location: East US 2 Comparing deployment state Comparing deployment state Creating/Updating resources You can view detailed progress in the Azure Portal: https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2F6e41a27a-b56f-423c-b5d9-bb0b325733eb%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdev-07030857-1719997066 (✓) Done: Resource group: rg-dev-07030857 (✓) Done: Key Vault: kv-7f4pci65jjxco (✓) Done: Storage account: st7f4pci65jjxco (✓) Done: Container Registry: cr7f4pci65jjxco ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: deploying to subscription: Deployment Error Details: UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. DeploymentActive: Unable to edit or replace deployment 'user-role-secrets-reader': previous deployment from '7/3/2024 8:57:51 AM' is still active (expiration time is '7/10/2024 8:57:49 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. DeploymentActive: Unable to edit or replace deployment 'ai': previous deployment from '7/3/2024 8:57:36 AM' is still active (expiration time is '7/10/2024 8:57:34 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. DeploymentActive: Unable to edit or replace deployment 'user-role-data-scientist': previous deployment from '7/3/2024 8:57:51 AM' is still active (expiration time is '7/10/2024 8:57:49 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. TraceID: 00000000000000000000000000000000
:x: azd down. [How to fix?] Error: Deleting all resources and deployed code on Azure (azd down) Local application code is not deleted when running 'azd down'. Initialize bicep provider Deleting your resources can take some time. Deleting resource group: rg-dev-07030857 (✓) Done: Deleting resource group: rg-dev-07030857 Purging Key Vault: kv-7f4pci65jjxco (x) Failed: Purging Key Vault: kv-7f4pci65jjxco ERROR: deleting infrastructure: error deleting Azure resources: purging resources: failed to purge Key Vault: purging key vault kv-7f4pci65jjxco: starting purging key vault: POST https://management.azure.com/subscriptions/6e41a27a-b56f-423c-b5d9-bb0b325733eb/providers/Microsoft.KeyVault/locations/eastus2/deletedVaults/kv-7f4pci65jjxco/purge -------------------------------------------------------------------------------- RESPONSE 404: 404 Not Found ERROR CODE: DeletedVaultNotFound -------------------------------------------------------------------------------- { "error": { "code": "DeletedVaultNotFound", "message": "The specified deleted vault 'kv-7f4pci65jjxco' does not exist. Ensure that the vault was indeed deleted and that it is in recoverable state. If soft delete was not enabled then the vault is permanently deleted. Follow this link for more information: https://go.microsoft.com/fwlink/?linkid=2149745" } } -------------------------------------------------------------------------------- TraceID: 00000000000000000000000000000000

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?] - Error: microsoft/security-devops-action is missing in .github/workflows/azure-dev.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/azure-dev.yml. - Error: microsoft/security-devops-action is missing in .github/workflows/bicep-audit.yml.
:warning: Security scan. [How to fix?] - error: AZR-000202 - By default, storage accounts accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. - error: AZR-000198 - Blob containers in Azure Storage Accounts can be configured for private or anonymous public access. By default, containers are private and only accessible with a credential or access token. When a container is configured with an access type other than private, anonymous access is permitted. Anonymous access to blobs or containers can be restricted by setting allowBlobPublicAccess to false. This enhanced security setting for a storage account overrides the individual settings for blob containers. When you disallow public access for a storage account, blobs are no longer accessible anonymously. - error: AZR-000355 - By default, Key Vault accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. If any of the following options are enabled you must also enable Allow trusted Microsoft services to bypass this firewall: - enabledForDeployment - Azure Virtual Machines for deployment. - enabledForDiskEncryption - Azure Disk Encryption for volume encryption. - enabledForTemplateDeployment - Azure Resource Manager for template deployment. - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: AZR-000291 - Every request to an Azure App Configuration resource must be authenticated. App Configuration supports authenticating requests using either Entra ID (previously Azure AD) identities or access keys. Using Entra ID identities: - Centralizes identity management and auditing. - Allows granting of permissions using role-based access control (RBAC). - Provides support for advanced security features such as conditional access and multi-factor authentication (MFA) when applicable. To require clients to use Entra ID to authenticate requests, you can disable the usage of access keys for an Azure App Configuration resource. When you disable access key authentication for an Azure App Configuration resource, any existing access keys for that resource are deleted. Any subsequent requests to the resource using the previously existing access keys will be rejected. Only requests that are authenticated using Entra ID will succeed. - error: AZR-000053 - API Management must authenticate to access Azure resources such as Key Vault. Use Key Vault to store certificates and secrets used within API Management. - error: AZR-000055 - API Management provides support for weak or deprecated ciphers. These older versions are provided for compatibility with clients and backends but are not consider secure. These many of these ciphers are enabled by default and need to be set to 'False'. The following ciphers are considered weak or deprecated: - TripleDes168 - TLS_RSA_WITH_AES_128_CBC_SHA - TLS_RSA_WITH_AES_256_CBC_SHA - TLS_RSA_WITH_AES_128_CBC_SHA256 - TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA - TLS_RSA_WITH_AES_256_CBC_SHA256 - TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA - TLS_RSA_WITH_AES_128_GCM_SHA256. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised. - error: AZR-000361 - Using managed identities have the following benefits: - Your app connects to resources with the managed identity. You don't need to manage credentials in your container app. - You can use role-based access control to grant specific permissions to a managed identity. - System-assigned identities are automatically created and managed. They're deleted when your container app is deleted. - You can add and delete user-assigned identities and assign them to multiple resources. They're independent of your container app's life cycle. - You can use managed identity to authenticate with a private Azure Container Registry without a username and password to pull containers for your Container App. - You can use managed identity to create connections for Dapr-enabled applications via Dapr components. - error: TA-000023 - To ensure that only applications from allowed networks, machines, or subnets can access your cluster, restrict access to your Kubernetes Service Management API server. It is recommended to limit access to authorized IP ranges to ensure that only applications from allowed networks can access the cluster. - error: AZR-000022 - To capture security-based audit logs from AKS clusters, the following diagnostic log categories should be enabled: - kube-audit or kube-audit-admin, or both. - kube-audit - Contains all audit log data for every audit event, including get, list, create, update, delete, patch, and post. - kube-audit-admin - Is a subset of the kube-audit log category. kube-audit-admin reduces the number of logs significantly by excluding the get and list audit events from the log. - guard - Contains logs for Azure Active Directory (AAD) authorization integration. For managed Azure AD, this includes token in and user info out. For Azure RBAC, this includes access reviews in and out. - error: AZR-000028 - AKS clusters support integration with Azure Policy using an Open Policy Agent (OPA). Azure Policy integration is provided by an optional add-on that can be enabled on AKS clusters. Once enabled and Azure policies assigned, AKS clusters will enforce the configured constraints. Examples of policies include: - Enforce HTTPS ingress in Kubernetes cluster. - Do not allow privileged containers in Kubernetes cluster. - Ensure container CPU and memory resource limits do not exceed the specified limits in Kubernetes cluster. - error: AZR-000029 - AKS-managed integration provides an easy way to use Azure AD authorization for AKS. Previous Azure AD integration with AKS required app registration and management within Azure AD. - error: AZR-000030 - In Kubernetes, the API server is the control plane of the cluster. Access to the API server is required by various cluster functions as well as all administrator activities. All activities performed against the cluster require authorization. To improve cluster security, the API server can be restricted to a limited set of IP address ranges. Restricting authorized IP addresses for the API server has the following limitations: - Requires AKS clusters configured with a Standard Load Balancer SKU. - This feature is not compatible with clusters that use Public IP per Node. - This feature is not compatible with AKS private clusters. When configuring this feature, you must specify the IP address ranges that will be authorized. To allow only the outbound public IP of the Standard SKU load balancer, use 0.0.0.0/32. You should add these ranges to the allow list: - Include output IP addresses for cluster nodes - Any range where administration will connect to the API server, including CI/CD systems, monitoring, and management systems. - error: AZR-000032 - Azure Kubernetes Service (AKS) supports Role-based Access Control (RBAC). RBAC is supported using Kubernetes RBAC and optionally Azure RBAC. - Using Kubernetes RBAC, you can grant users, groups, and service accounts access to cluster resources. - Additionally AKS supports granting Azure AD identities access to cluster resources using Azure RBAC. Using authorization provided by Azure RBAC simplifies and centralizes authorization of Azure AD principals. Access to Kubernetes resource can be managed using Azure Resource Manager (ARM). When Azure RBAC is enabled: - Azure AD principals will be validated exclusively by Azure RBAC. - Kubernetes users and service accounts are exclusively validated by Kubernetes RBAC. - error: AZR-000033 - AKS clusters may need to store and retrieve secrets, keys, and certificates. The Secrets Store CSI Driver provides cluster support to integrate with Key Vault. When enabled and configured secrets, keys, and certificates can be securely accessed from a pod. The Secrets Store CSI Driver can automatically refresh secrets and keys periodically from Key Vault. To enable this feature, enable Secrets Store CSI Driver autorotation. Avoid storing secrets to access Azure resources. Use a Managed Identity when possible instead of cryptographic keys or a regular service principal. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: AZR-000095 - Cosmos DB provides two authorization options for interacting with the database: - Azure Active Directory identity (Azure AD). Can be used to authorize account and resource management operations. - Keys and resource tokens. Can be used to authorize resource management and data operations. Resource management operations include management of databases, indexes, and containers. By default, keys are permitted to perform resource management operations. You can restrict these operations to Azure Resource Manager (ARM) calls only. - error: AZR-000186 - Enable Microsoft Defender for Azure SQL logical server. - error: AZR-000187 - Auditing for Azure SQL Database tracks database events and writes them to an audit log. Audit logs help you find suspicious events, unusual activity, and trends. - error: AZR-000188 - Azure SQL Database offer two authentication models, Azure Active Directory (AAD) and SQL authentication. AAD authentication supports centralized identity management in addition to modern password protections. Some of the benefits of AAD authentication over SQL authentication including: - Support for Azure Multi-Factor Authentication (MFA). - Conditional-based access with Conditional Access. It is also possible to disable SQL authentication entirely and only use AAD authentication. - error: AZR-000316 - Azure Bicep and Azure Resource Manager (ARM) templates can be used to deploy resources to Azure. When deploying Azure resources, sensitive values such as passwords, certificates, and keys should be passed as secure parameters. Secure parameters use the secureString or secureObject type. Parameters that do not use secure types are recorded in logs and deployment history. These values can be retrieved by anyone with access to the deployment history. - error: AZR-000316 - Azure Bicep and Azure Resource Manager (ARM) templates can be used to deploy resources to Azure. When deploying Azure resources, sensitive values such as passwords, certificates, and keys should be passed as secure parameters. Secure parameters use the secureString or secureObject type. Parameters that do not use secure types are recorded in logs and deployment history. These values can be retrieved by anyone with access to the deployment history. - error: AZR-000316 - Azure Bicep and Azure Resource Manager (ARM) templates can be used to deploy resources to Azure. When deploying Azure resources, sensitive values such as passwords, certificates, and keys should be passed as secure parameters. Secure parameters use the secureString or secureObject type. Parameters that do not use secure types are recorded in logs and deployment history. These values can be retrieved by anyone with access to the deployment history. - error: AZR-000316 - Azure Bicep and Azure Resource Manager (ARM) templates can be used to deploy resources to Azure. When deploying Azure resources, sensitive values such as passwords, certificates, and keys should be passed as secure parameters. Secure parameters use the secureString or secureObject type. Parameters that do not use secure types are recorded in logs and deployment history. These values can be retrieved by anyone with access to the deployment history. - warning: AZR-000388 - Azure RBAC is the recommended authorization system for the Azure Key Vault data plane. Azure RBAC allows users to manage key, secrets, and certificates permissions. It provides one place to manage all permissions across all Key Vaults. Azure RBAC for Key Vault also allows users to have separate permissions on individual keys, secrets, and certificates. The Azure RBAC permission model is not enabled by default. - warning: AZR-000406 - Disabling public network access improves security by ensuring that the Machine Learning Workspaces aren't exposed on the public internet. You can control exposure of your workspaces by creating private endpoints instead. By default, a public endpoint is enabled for Machine Learning workspaces. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help control exposure of a workspace to data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - warning: AZR-000407 - Manange access to Azure ML workspace and associated resources, Azure Container Registry, KeyVault, Storage, and App Insights using user-assigned managed identity. By default, system-assigned managed identity is used by Azure ML workspace to access the associated resources. User-assigned managed identity allows you to create the identity as an Azure resource and maintain the life cycle of that identity. - warning: AZR-000406 - Disabling public network access improves security by ensuring that the Machine Learning Workspaces aren't exposed on the public internet. You can control exposure of your workspaces by creating private endpoints instead. By default, a public endpoint is enabled for Machine Learning workspaces. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help control exposure of a workspace to data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - warning: AZR-000407 - Manange access to Azure ML workspace and associated resources, Azure Container Registry, KeyVault, Storage, and App Insights using user-assigned managed identity. By default, system-assigned managed identity is used by Azure ML workspace to access the associated resources. User-assigned managed identity allows you to create the identity as an Azure resource and maintain the life cycle of that identity. - warning: AZR-000031 - AKS clusters support Role-based Access Control (RBAC) authorization. RBAC allows users, groups, and service accounts to be granted access to resources on an as needed basis. Actions performed by each identity can be logged for auditing with Kubernetes audit policies. When a cluster is deployed, local accounts are enabled by default even when RBAC is enabled. These local accounts such as clusterAdmin and clusterUser are shared accounts that are not tied to an identity. If local account credentials are used, Kubernetes auditing logs the local account instead of named accounts. Who performed an action cannot be determined from the audit logs, creating an audit log gap for privileged actions. In an AKS cluster with local account disabled administrator will be unable to get the clusterAdmin credential. For example, using az aks get-credentials -g '' -n '' --admin will fail. - warning: AZR-000370 - To collect and provide data plane protections of Microsoft Defender for Containers some additional daemon set and deployments needs to be deployed to the AKS clusters. These components are installed when the Defender profile is enabled on the cluster. The Defender profile deployed to each node provides the runtime protections and collects signals from nodes. - warning: AZR-000390 - Azure Database for PostgreSQL supports authentication with PostgreSQL logins and Azure AD authentication. By default, authentication with PostgreSQL logins is enabled. PostgreSQL logins are unable to provide sufficient protection for identities. Azure AD authentication provides strong protection controls including conditional access, identity governance, and privileged identity management. Once you decide to use Azure AD authentication, you can disable authentication with PostgreSQL logins. Azure AD-only authentication is only supported for the flexible server deployment model.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/openai-chat-app-quickstart

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Getting Started is missing in README.md. - Error: ## Guidance is missing in README.md.
:x: LICENSE File. [How to fix?] - Error: LICENSE file is missing.
:heavy_check_mark: SECURITY.md File.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:x: CONTRIBUTING.md File. [How to fix?] - Error: CONTRIBUTING.md file is missing.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:heavy_check_mark: Topics on repo contains ['azd-templates', 'ai-azd-templates'].

Source code structure and conventions:

:x: .github/workflows/azure-dev.yml File. [How to fix?] - Error: .github/workflows/azure-dev.yml file is missing.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Packaging services (azd package) Packaging service aca Packaging service aca (Tagging container image) (✓) Done: Packaging service aca - Image Hash: sha256:f0fbc90b3ee7ac163ebb3cfe4516ba4941d4c23e082cdd596d7cd84e5ab4af42 - Target Image: simple-chatgpt-python/aca-dev-07030856:azd-deploy-1719997054 Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time. Initialize bicep provider Retrieving subscriptions... Reading subscription and location from environment... Subscription: Visual Studio Enterprise Subscription (6e41a27a-b56f-423c-b5d9-bb0b325733eb) Location: East US 2 Creating a deployment plan Comparing deployment state Creating/Updating resources You can view detailed progress in the Azure Portal: https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2F6e41a27a-b56f-423c-b5d9-bb0b325733eb%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdev-07030856-1719997060 (✓) Done: Resource group: dev-07030856-rg (✓) Done: Azure OpenAI: fq7hgyla7b2aw-cog (✓) Done: Log Analytics workspace: dev-07030856-fq7hgyla7b2aw-loganalytics (✓) Done: Container Registry: dev07030856fq7hgyla7b2awregistry (✓) Done: Container Apps Environment: dev-07030856-fq7hgyla7b2aw-containerapps-env (✓) Done: Container App: dev-07030856-fq7hgy-ca ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: deploying to subscription: Deployment Error Details: UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. TraceID: 00000000000000000000000000000000
:heavy_check_mark: azd down.

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?]
:warning: Security scan. [How to fix?] - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: AZR-000361 - Using managed identities have the following benefits: - Your app connects to resources with the managed identity. You don't need to manage credentials in your container app. - You can use role-based access control to grant specific permissions to a managed identity. - System-assigned identities are automatically created and managed. They're deleted when your container app is deleted. - You can add and delete user-assigned identities and assign them to multiple resources. They're independent of your container app's life cycle. - You can use managed identity to authenticate with a private Azure Container Registry without a username and password to pull containers for your Container App. - You can use managed identity to create connections for Dapr-enabled applications via Dapr components. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/openai-plugin-fastapi

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Features is missing in README.md. - Error: ## Getting Started is missing in README.md. - Error: ## Guidance is missing in README.md. - Error: ## Resources is missing in README.md.
:heavy_check_mark: LICENSE File.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:x: .github/CODE_OF_CONDUCT.md File. [How to fix?] - Error: .github/CODE_OF_CONDUCT.md file is missing.
:x: CONTRIBUTING.md File. [How to fix?] - Error: CONTRIBUTING.md file is missing.
:x: .github/ISSUE_TEMPLATE.md File. [How to fix?] - Error: .github/ISSUE_TEMPLATE.md file is missing.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:heavy_check_mark: .github/workflows/azure-dev.yml File.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Packaging services (azd package) Packaging service api Packaging service api (Tagging container image) (✓) Done: Packaging service api - Image Hash: sha256:26c0940f904431ca3233c08904613798dd20ad1eed13dc4b64ee8874b9e13f41 - Target Image: openai-plugin-fastapi/api-dev-07030857:azd-deploy-1719997077 Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time. Initialize bicep provider Retrieving subscriptions... Reading subscription and location from environment... Subscription: Visual Studio Enterprise Subscription (6e41a27a-b56f-423c-b5d9-bb0b325733eb) Location: East US 2 Comparing deployment state Comparing deployment state Creating/Updating resources You can view detailed progress in the Azure Portal: https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2F6e41a27a-b56f-423c-b5d9-bb0b325733eb%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdev-07030857-1719997082 (✓) Done: Resource group: rg-dev-07030857 (✓) Done: Log Analytics workspace: loganalytics ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: deploying to subscription: Deployment Error Details: DeploymentActive: Unable to edit or replace deployment 'monitoring': previous deployment from '7/3/2024 8:58:00 AM' is still active (expiration time is '7/10/2024 8:57:59 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. TraceID: 00000000000000000000000000000000
:x: azd down. [How to fix?] Error: Deleting all resources and deployed code on Azure (azd down) Local application code is not deleted when running 'azd down'. Initialize bicep provider Deleting your resources can take some time. Deleting resource group: rg-dev-07030857 (✓) Done: Deleting resource group: rg-dev-07030857 Purging Key Vault: kv-7f4pci65jjxco (x) Failed: Purging Key Vault: kv-7f4pci65jjxco ERROR: deleting infrastructure: error deleting Azure resources: purging resources: failed to purge Key Vault: purging key vault kv-7f4pci65jjxco: starting purging key vault: POST https://management.azure.com/subscriptions/6e41a27a-b56f-423c-b5d9-bb0b325733eb/providers/Microsoft.KeyVault/locations/eastus2/deletedVaults/kv-7f4pci65jjxco/purge -------------------------------------------------------------------------------- RESPONSE 404: 404 Not Found ERROR CODE: DeletedVaultNotFound -------------------------------------------------------------------------------- { "error": { "code": "DeletedVaultNotFound", "message": "The specified deleted vault 'kv-7f4pci65jjxco' does not exist. Ensure that the vault was indeed deleted and that it is in recoverable state. If soft delete was not enabled then the vault is permanently deleted. Follow this link for more information: https://go.microsoft.com/fwlink/?linkid=2149745" } } -------------------------------------------------------------------------------- TraceID: 00000000000000000000000000000000

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?] - Error: microsoft/security-devops-action is missing in .github/workflows/azure-dev.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/azure-dev.yml.
:warning: Security scan. [How to fix?] - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: AZR-000005 - Azure Container Registry (ACR) includes a built-in local admin user account. The admin user account is a single user account with administrative access to the registry. This account provides single user access for early test and development. The admin user account is not intended for use with production container registries. Instead of using the admin user account, consider using Entra ID (previously Azure AD) identities. Entra ID provides a centralized identity and authentication system for Azure. This provides a number of benefits including: - Strong account protection controls with conditional access, identity governance, and privileged identity management. - Auditing and reporting of account activity. - Granular access control with role-based access control (RBAC). - Separation of account types for users and applications. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/rag-postgres-openai-python

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Getting Started is missing in README.md. - Error: ## Guidance is missing in README.md.
:x: LICENSE File. [How to fix?] - Error: LICENSE file is missing.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:heavy_check_mark: Topics on repo contains ['azd-templates', 'ai-azd-templates'].

Source code structure and conventions:

:x: .github/workflows/azure-dev.yml File. [How to fix?] - Error: .github/workflows/azure-dev.yml file is missing.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Retrieving locations... ERROR: prompting for value: prompting for location: no default response for prompt 'Enter a value for the 'openAILocation' infrastructure parameter:'
:x: azd down. [How to fix?] Error: Deleting all resources and deployed code on Azure (azd down) Local application code is not deleted when running 'azd down'. Initialize bicep provider Retrieving locations... Enter a value for the 'openAILocation' infrastructure parameter: ERROR: initializing provisioning manager: prompting for value: prompting for location: '' is not an allowed choice. allowed choices: 1. (Canada) Canada East (canadaeast), 2. (US) North Central US (northcentralus), 3. (US) South Central US (southcentralus)

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?]
:warning: Security scan. [How to fix?] - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: AZR-000361 - Using managed identities have the following benefits: - Your app connects to resources with the managed identity. You don't need to manage credentials in your container app. - You can use role-based access control to grant specific permissions to a managed identity. - System-assigned identities are automatically created and managed. They're deleted when your container app is deleted. - You can add and delete user-assigned identities and assign them to multiple resources. They're independent of your container app's life cycle. - You can use managed identity to authenticate with a private Azure Container Registry without a username and password to pull containers for your Container App. - You can use managed identity to create connections for Dapr-enabled applications via Dapr components. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/serverless-chat-langchainjs

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Features is missing in README.md. - Error: ## Getting Started is missing in README.md. - Error: ## Guidance is missing in README.md.
:heavy_check_mark: LICENSE File.
:heavy_check_mark: SECURITY.md File.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:x: .github/workflows/azure-dev.yml File. [How to fix?] - Error: .github/workflows/azure-dev.yml file is missing.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Packaging services (azd package) Packaging service api Packaging service api (Installing NPM dependencies) Packaging service api (Running NPM package script) Packaging service api (Copying deployment package) Packaging service api (Compressing deployment artifacts) (✓) Done: Packaging service api - Package Output: /tmp/serverless-chat-langchainjs-api-azddeploy-1719997050.zip Packaging service webapp Packaging service webapp (Installing NPM dependencies) Packaging service webapp (Running NPM package script) Packaging service webapp (Copying deployment package) (✓) Done: Packaging service webapp - Package Output: dist Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time. Initialize bicep provider Retrieving subscriptions... Reading subscription and location from environment... Subscription: Visual Studio Enterprise Subscription (6e41a27a-b56f-423c-b5d9-bb0b325733eb) Location: East US 2 Creating a deployment plan Comparing deployment state Creating/Updating resources You can view detailed progress in the Azure Portal: https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2F6e41a27a-b56f-423c-b5d9-bb0b325733eb%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdev-07030856-1719997062 (✓) Done: Resource group: rg-dev-07030856 (✓) Done: App Service plan: plan-fq7hgyla7b2aw (✓) Done: Storage account: stfq7hgyla7b2aw (✓) Done: Azure OpenAI: cog-fq7hgyla7b2aw (x) Failed: Search service: srch-fq7hgyla7b2aw ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: deploying to subscription: Deployment Error Details: InvalidTemplateDeployment: The template deployment 'webapp' is not valid according to the validation procedure. The tracking id is 'bf3dccc8-e9ad-4e68-994f-864d807d0999'. See inner errors for details. ValidationForResourceFailed: Validation failed for a resource. Check 'Error.Details[0]' for more information. InvalidSkuName: Free SKU is invalid. Managed service identities are not allowed in this SKU. UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. UnmatchedPrincipalType: The PrincipalId '95959d96ca17490a877b63300e2f9187' has type 'ServicePrincipal' , which is different from specified PrinciaplType 'User'. DeploymentActive: Unable to edit or replace deployment 'storage': previous deployment from '7/3/2024 8:57:49 AM' is still active (expiration time is '7/10/2024 8:57:42 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. DeploymentActive: Unable to edit or replace deployment 'openai': previous deployment from '7/3/2024 8:57:46 AM' is still active (expiration time is '7/10/2024 8:57:42 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. TraceID: 00000000000000000000000000000000
:heavy_check_mark: azd down.

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?] - Error: microsoft/security-devops-action is missing in .github/workflows/stale-bot.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/stale-bot.yml.
:warning: Security scan. [How to fix?] - error: AZR-000202 - By default, storage accounts accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. - error: AZR-000198 - Blob containers in Azure Storage Accounts can be configured for private or anonymous public access. By default, containers are private and only accessible with a credential or access token. When a container is configured with an access type other than private, anonymous access is permitted. Anonymous access to blobs or containers can be restricted by setting allowBlobPublicAccess to false. This enhanced security setting for a storage account overrides the individual settings for blob containers. When you disallow public access for a storage account, blobs are no longer accessible anonymously. - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised. - error: TA-000001 - Enable auditing of diagnostic logs on the app. This enables you to recreate activity trails for investigation purposes if a security incident occurs or your network is compromised.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/summarization-openai-csharp-prompty

Repository Management:

:heavy_check_mark: README.md File.
:heavy_check_mark: LICENSE File.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: azd-templates is missing in topics. - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:heavy_check_mark: .github/workflows/azure-dev.yml File.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Retrieving locations... Retrieving locations... Packaging services (azd package) Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time. Initialize bicep provider Retrieving subscriptions... Reading subscription and location from environment... Subscription: Visual Studio Enterprise Subscription (6e41a27a-b56f-423c-b5d9-bb0b325733eb) Location: East US 2 Comparing deployment state Comparing deployment state Creating/Updating resources You can view detailed progress in the Azure Portal: https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2F6e41a27a-b56f-423c-b5d9-bb0b325733eb%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdev-07030857-1719997038 (✓) Done: Resource group: rg-dev-07030857 (✓) Done: Resource group: rg-dev-07030857 (✓) Done: Cognitive Service: cog-sp-7f4pci65jjxco (✓) Done: Azure OpenAI: 7f4pci65jjxco-cog (✓) Done: Log Analytics workspace: loganalytics (✓) Done: Container Registry: dev070308577f4pci65jjxcoregistry (✓) Done: Container Apps Environment: dev-07030857-7f4pci65jjxco-containerapps-env ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: deploying to subscription: Deployment Error Details: ResourceGroupBeingDeleted: The resource group 'rg-dev-07030857' is in deprovisioning state and cannot perform this operation. TraceID: 00000000000000000000000000000000
:heavy_check_mark: azd down.

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?] - Error: microsoft/security-devops-action is missing in .github/workflows/azure-dev.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/azure-dev.yml. - Error: microsoft/security-devops-action is missing in .github/workflows/Evaluation.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/Evaluation.yml.
:warning: Security scan. [How to fix?] - error: AZR-000355 - By default, Key Vault accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. If any of the following options are enabled you must also enable Allow trusted Microsoft services to bypass this firewall: - enabledForDeployment - Azure Virtual Machines for deployment. - enabledForDiskEncryption - Azure Disk Encryption for volume encryption. - enabledForTemplateDeployment - Azure Resource Manager for template deployment. - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: AZR-000361 - Using managed identities have the following benefits: - Your app connects to resources with the managed identity. You don't need to manage credentials in your container app. - You can use role-based access control to grant specific permissions to a managed identity. - System-assigned identities are automatically created and managed. They're deleted when your container app is deleted. - You can add and delete user-assigned identities and assign them to multiple resources. They're independent of your container app's life cycle. - You can use managed identity to authenticate with a private Azure Container Registry without a username and password to pull containers for your Container App. - You can use managed identity to create connections for Dapr-enabled applications via Dapr components. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: AZR-000095 - Cosmos DB provides two authorization options for interacting with the database: - Azure Active Directory identity (Azure AD). Can be used to authorize account and resource management operations. - Keys and resource tokens. Can be used to authorize resource management and data operations. Resource management operations include management of databases, indexes, and containers. By default, keys are permitted to perform resource management operations. You can restrict these operations to Azure Resource Manager (ARM) calls only. - warning: AZR-000388 - Azure RBAC is the recommended authorization system for the Azure Key Vault data plane. Azure RBAC allows users to manage key, secrets, and certificates permissions. It provides one place to manage all permissions across all Key Vaults. Azure RBAC for Key Vault also allows users to have separate permissions on individual keys, secrets, and certificates. The Azure RBAC permission model is not enabled by default.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.

AI Gallery Standard Validation: FAILED for Azure-Samples/summarization-openai-python-promptflow

Repository Management:

:x: README.md File. [How to fix?] - Error: ## Getting Started is missing in README.md. - Error: ## Guidance is missing in README.md.
:heavy_check_mark: LICENSE File.
:x: SECURITY.md File. [How to fix?] - Error: SECURITY.md file is missing.
:heavy_check_mark: .github/CODE_OF_CONDUCT.md File.
:heavy_check_mark: CONTRIBUTING.md File.
:heavy_check_mark: .github/ISSUE_TEMPLATE.md File.
:x: Topics on repo contains ['azd-templates', 'ai-azd-templates']. [How to fix?] - Error: azd-templates is missing in topics. - Error: ai-azd-templates is missing in topics.

Source code structure and conventions:

:heavy_check_mark: .github/workflows/azure-dev.yml File.
:x: .github/workflows/pr-gate.yml File. [How to fix?] - Error: .github/workflows/pr-gate.yml file is missing.
:heavy_check_mark: azure.yaml File.
:heavy_check_mark: infra Folder.
:heavy_check_mark: .devcontainer Folder.

Functional Requirements:

:x: azd up. [How to fix?] Error: Downloading Bicep (✓) Done: Downloading Bicep Initialize bicep provider Retrieving locations... Retrieving locations... Packaging services (azd package) Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time. Initialize bicep provider Retrieving subscriptions... Reading subscription and location from environment... Subscription: Visual Studio Enterprise Subscription (6e41a27a-b56f-423c-b5d9-bb0b325733eb) Location: East US 2 Creating a deployment plan Comparing deployment state Creating/Updating resources You can view detailed progress in the Azure Portal: https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2F6e41a27a-b56f-423c-b5d9-bb0b325733eb%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fdev-07030857-1719997047 (✓) Done: Resource group: rg-dev-07030857 (✓) Done: Resource group: rg-dev-07030857 (✓) Done: Cognitive Service: cog-sp-7f4pci65jjxco (✓) Done: Azure OpenAI: 7f4pci65jjxco-cog (✓) Done: Log Analytics workspace: loganalytics (✓) Done: Container Registry: dev070308577f4pci65jjxcoregistry (✓) Done: Container Apps Environment: dev-07030857-7f4pci65jjxco-containerapps-env ERROR: error executing step command 'provision': deployment failed: error deploying infrastructure: deploying to subscription: Deployment Error Details: ResourceGroupBeingDeleted: The resource group 'rg-dev-07030857' is in deprovisioning state and cannot perform this operation. ResourceGroupBeingDeleted: The resource group 'rg-dev-07030857' is in deprovisioning state and cannot perform this operation. DeploymentActive: Unable to edit or replace deployment 'speechRecognizer': previous deployment from '7/3/2024 8:57:27 AM' is still active (expiration time is '7/10/2024 8:57:24 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. DeploymentActive: Unable to edit or replace deployment 'openai': previous deployment from '7/3/2024 8:57:27 AM' is still active (expiration time is '7/10/2024 8:57:25 AM'). Please see https://aka.ms/arm-deploy-resources for usage details. TraceID: 00000000000000000000000000000000
:heavy_check_mark: azd down.

Security Requirements:

:x: microsoft/security-devops-action is integrated to the CI/CD pipeline. [How to fix?] - Error: microsoft/security-devops-action is missing in .github/workflows/azure-dev.yml. - Error: github/codeql-action/upload-sarif is missing in .github/workflows/azure-dev.yml.
:warning: Security scan. [How to fix?] - error: AZR-000355 - By default, Key Vault accept connections from clients on any network. To limit access to selected networks, you must first change the default action. After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: - Azure services on the trusted service list. - IP address or CIDR range. - Private endpoint connections. - Azure virtual network subnets with a Service Endpoint. If any of the following options are enabled you must also enable Allow trusted Microsoft services to bypass this firewall: - enabledForDeployment - Azure Virtual Machines for deployment. - enabledForDiskEncryption - Azure Disk Encryption for volume encryption. - enabledForTemplateDeployment - Azure Resource Manager for template deployment. - error: AZR-000280 - By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated. Configure service endpoints and private links where appropriate. - error: AZR-000282 - To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits. With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing. Once you decide to use Azure AD authentication, you can disable authentication using keys. - warning: AZR-000283 - By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks. Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required. - error: AZR-000361 - Using managed identities have the following benefits: - Your app connects to resources with the managed identity. You don't need to manage credentials in your container app. - You can use role-based access control to grant specific permissions to a managed identity. - System-assigned identities are automatically created and managed. They're deleted when your container app is deleted. - You can add and delete user-assigned identities and assign them to multiple resources. They're independent of your container app's life cycle. - You can use managed identity to authenticate with a private Azure Container Registry without a username and password to pull containers for your Container App. - You can use managed identity to create connections for Dapr-enabled applications via Dapr components. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: AZR-000363 - Container apps environments allows you to expose your container app to the Internet. Container apps environments deployed as external resources are available for public requests. External environments are deployed with a virtual IP on an external, public facing IP address. Disable public network access to improve security by exposing the Container Apps environment through an internal load balancer. This removes the need for a public IP address and prevents internet access to all Container Apps within the environment. To provide secure access, instead consider using an Application Gateway or Azure Front Door premium in front of your Container Apps on your private VNET. - error: AZR-000095 - Cosmos DB provides two authorization options for interacting with the database: - Azure Active Directory identity (Azure AD). Can be used to authorize account and resource management operations. - Keys and resource tokens. Can be used to authorize resource management and data operations. Resource management operations include management of databases, indexes, and containers. By default, keys are permitted to perform resource management operations. You can restrict these operations to Azure Resource Manager (ARM) calls only. - warning: AZR-000388 - Azure RBAC is the recommended authorization system for the Azure Key Vault data plane. Azure RBAC allows users to manage key, secrets, and certificates permissions. It provides one place to manage all permissions across all Key Vaults. Azure RBAC for Key Vault also allows users to have separate permissions on individual keys, secrets, and certificates. The Azure RBAC permission model is not enabled by default.

How to fix?

The full Definition of Done of the AI-Gallery template and fix approached can be found HERE.