search

Azure / bicep

Bicep is a declarative language for describing and deploying Azure resources
MIT License
3.21k stars 745 forks source link

MS Graph (AAD) provider for bicep #7724

Closed spmanjunath closed 4 months ago

spmanjunath commented 2 years ago

There appears to be no clean way of registering a new application (App registration) in Azure AD using Bicep. The suggested alternate approach involves using,

For the above approach, the User identity should be created beforehand which defeats the purpose (of enveloping everything related to app deployment in a Bicep file) as it need to be created manually, or, through Powershell scripts.

I see a similar request as part of ARM, was wondering if this feature can be considered for future implementation by Bicep team.

Note: It appears that Terraform supports similar requirement.

alex-frankel commented 2 years ago

We are working on an MS Graph (AAD) provider for Bicep so you can create App registrations and other AAD objects, but don't have a clear ETA atm.

Related to #6864

cwe1ss commented 1 year ago

We are working on an MS Graph (AAD) provider for Bicep so you can create App registrations and other AAD objects, but don't have a clear ETA atm.

Related to #6864

This is fantastic news. Will this also allow us to reference/query existing AAD objects? (E.g. to set the SQL Server AAD admin to an existing AAD group)

alex-frankel commented 1 year ago

We are working on an MS Graph (AAD) provider for Bicep so you can create App registrations and other AAD objects, but don't have a clear ETA atm. Related to #6864

This is fantastic news. Will this also allow us to reference/query existing AAD objects? (E.g. to set the SQL Server AAD admin to an existing AAD group)

Yes, we will support existing references for AAD objects, which will be semantically equivalent to existing for Azure resources.

Agger1995 commented 1 year ago

I am very happy to have found this issue, and hope to see it implemented at some point. This would be such a huge quality of life update for us! We are currently limited to manually creating certain AAD resources prior to deploying our Azure infrastructure. Do you know whether the MS Graph provider will have support for creating group assignments?

alex-frankel commented 1 year ago

Yes -- service principal, group, and group assignment CRUD are the three Graph objects we are focused on in the first iteration. It's possible we will be able to support more than that, but those three are the priority.

Sdelausnay commented 1 year ago

Hi,

Also following this tread. Is there already an estimated time this will be released?

ChristopherGLewis commented 1 year ago

I've got a demo in GitHub of how to use DeploymentScripts in bicep - the sample uses a PowerShell script to grab a Role Definition's GUID, but with minimal effort this could be converted to your usage scenario.

BicepDeploymentScripts

It basically creates a User Assigned Identity, grants the UAI reader to the subscription, assigns that to the DeploymentScript, and then finally runs a PowerShell script to query Azure to get the specified role's GUID.

vinvli commented 1 year ago

Hi,

Also following this tread. Is there already an estimated time this will be released?

Also very interested in knowing when this will be available. We want to use it to create Azure Service Bus queues and create/assign the appropriate group/ service principal to it using bicep.

alex-frankel commented 1 year ago

We are still closing on some design details, so we can't give a confident ETA yet. Will share more details as we have them.

Grant-Rc commented 1 year ago

This would be great to support the azure landing zones that we are implanting atm. Can do alot in the bicep modules apart from created the AAD groups for the RBAK roles

josdeweger commented 1 year ago

We have worked around this for now using a deployment script, but really keen to be able to entirely do this in Bicep. Any news?

ms-sambell commented 1 year ago

Hey team,

This was discussed in the last Bicep meeting. When is it going live?

alex-frankel commented 1 year ago

Last ETA I heard was September. @stephaniezyen can confirm.

MaluSiv commented 1 year ago

Following this thread.Any ETA?

NSimpragaVolur commented 1 year ago

Also following this. I think anyone using Bicep is extremely keen on having support for managing AAD with it too.

alex-frankel commented 1 year ago

ETA is 9/15.

@NsimpragaVolur - it's my understanding that MSGraph/AAD/EntraID are all synonyms, but I could have that wrong. Is there specific functionality you are expecting that is not captured on this issue?

pamelafox commented 1 year ago

(Personally) I am hoping to replace these scripts with Bicep:

https://github.com/microsoft/sample-app-aoai-chatGPT/blob/main/scripts/auth_init.py https://github.com/microsoft/sample-app-aoai-chatGPT/blob/main/scripts/auth_update.py

It uses the REST API to create a Graph application, add a client secret, and store the resulting app ID / client ID / client secret (to feed into Bicep). It then updates redirect URI.

So I am hoping the 9/15 Bicep will enable all that.

SvenAelterman commented 1 year ago

(Personally) I am hoping to replace these scripts with Bicep:

https://github.com/microsoft/sample-app-aoai-chatGPT/blob/main/scripts/auth_init.py https://github.com/microsoft/sample-app-aoai-chatGPT/blob/main/scripts/auth_update.py

It uses the REST API to create a Graph application, add a client secret, and store the resulting app ID / client ID / client secret (to feed into Bicep). It then updates redirect URI.

So I am hoping the 9/15 Bicep will enable all that.

That would be my # 1 use case too.

In addition, looking up the object ID of a user or group by UPN or name.

jikuja commented 1 year ago

From monthly call: image (I would have posted this earlier but missed capturing screenshot from live presentation)

The demo on monthly was really good: 5/5 probably will check again soon.

About MS Graph/Entra something/Azure AD/etc:

dkershaw10 commented 1 year ago

Just to recap - capabilities and limitations for private preview:

@pamelafox - yes that should be possible through a Bicep template alone. @SvenAelterman - we won't have support for user as a bicep type in the private preview, but we can add that later. You'll be able to reference groups you create via a client provided key (just like Azure resources).

Hope this helps.

dkershaw10 commented 1 year ago

Here's a very simple example: image

NSimpragaVolur commented 1 year ago

@alex-frankel When I say AAD functionality I mean being able to reference and create AAD types, primarily Security Groups, App Registrations and Service Principals/Enterprise applications.

franklixuefei commented 1 year ago

@alex-frankel just wanted to get an idea, when bicep officially supports this, is it just adding more types for Microsoft.Identity RP, or it is fundamentally different than what Microsoft.Identity RP does? As of today, Microsoft.Identity supports full AAD app/serviceprincipal buildout as per the doc, but only under Azure infra tenants (xME tenants). Will bicep support all tenants?

alex-frankel commented 1 year ago

@franklixuefei - the Microsft.Identity RP is only internal-facing. The MS Graph provider is the functional replacement for this provider for both internal and external users.

shaulimaya commented 9 months ago

So any update when and how i can create app registration for web auth using bicep?

alex-frankel commented 9 months ago

Right now there is a private preview running, but last I heard signups are closed until a Private Preview refresh planned for early next year. cc @dkershaw10 as FYI.

shaulimaya commented 9 months ago

I am creating azure managed app using biceps So can you suggest how can I create a function with ms provider for auth? I need to create the registered application with the right web redirect url? I think that the ama has no permissions for creating register web app in the tenant...even if i will use the scripts.. Am i missing something?

alex-frankel commented 9 months ago

In order to do MS Graph operations, like create an App Registration, as part of a bicep deployment, there are now two options:

franklixuefei commented 9 months ago

3rd option for internal folks - use the Microsoft.Identity RP!

On Tue, Dec 5, 2023 at 4:10 PM Alex Frankel @.***> wrote:

In order to do MS Graph operations like create an App Registration as part of a bicep deployment, there are now two options:

— Reply to this email directly, view it on GitHub https://github.com/Azure/bicep/issues/7724#issuecomment-1841858151 or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKF5RTYSPLMSMSLYR4B62DYH6ZXXBFKMF2HI4TJMJ2XIZLTSWBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLAVFOZQWY5LFVIZDANJWHAZTMOJUGOSG4YLNMWUWQYLTL5WGCYTFNSBKK5TBNR2WLKRSHE4TIMZUG42DMMVENZQW2ZNJNBQXGX3MMFRGK3FMON2WE2TFMN2F65DZOBS2YSLTON2WKQ3PNVWWK3TUUZ2G64DJMNZZJAVEOR4XAZNKOJSXA33TNF2G64TZUV3GC3DVMWUTENRTGUYDGMRVGCBKI5DZOBS2K2LTON2WLJLWMFWHKZNKGEZTCOJVGI3TKNRQQKSHI6LQMWSWYYLCMVWKK5TBNR2WLKRSGA2TMOBTGY4TIM4CUR2HS4DFUVWGCYTFNSSXMYLMOVS2UMRZHE2DGNBXGQ3DFJ3UOJUWOZ3FOKTGG4TFMF2GK . You are receiving this email because you commented on the thread.

Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

aucampia commented 9 months ago

This option would go against various recommendations from Microsoft that IaC should be declarative:

https://learn.microsoft.com/en-us/devops/deliver/what-is-infrastructure-as-code#use-declarative-definition-files

IaC should use declarative definition files if possible.

https://learn.microsoft.com/en-us/azure/well-architected/operational-excellence/infrastructure-as-code-design

Prefer declarative over imperative tools. Declarative tools and their associated files are a better overall choice for deploying and managing IaC than imperative tools.

https://microsoft.github.io/azureml-ops-accelerator/1-MLOpsFoundation/0-DevOpsOverview/9-IaaC.html#prefer-declarative-definitions

The preferred approach to IaC is to use declarative definition files where possible.

And it is possible to use declarative definition files for this:

alex-frankel commented 9 months ago

Hi @aucampia - I was only trying to spell out the options to accomplish this with Bicep today. We certainly agree that having to fallback to Deployment Scripts is not an ideal option. That is why we are working on the MS Graph provider..

Of course, the other tools you mentioned are also great options to use in conjunction with or as an alternative to bicep.

jongio commented 8 months ago

I believe the new link is:

https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/deployment-script-develop?tabs=CLI#use-microsoft-graph-within-a-deployment-script

Keilor2019 commented 6 months ago

Is this bicep template going to be available for Azure AD B2C ? To create app registrations there for web authentication using MSAL.

Bandgren commented 6 months ago

@dkershaw10 & @alex-frankel How is the private preview going? We have a super nice use case for what the initial provider is supporting and we are thinking of maybe pushing our goal until may if the public preview will be public by then. Or has it been pushed back?

frankz217 commented 6 months ago

@alex-frankel Wondering about the progress as well!

kdambekalns commented 6 months ago

Right now I am at the point I'd need this… bummer it's only scheduled for May!

huajunzhao commented 6 months ago

@alex-frankel

Is Graph provider available now for internal use? If yes, could you please share the guidance?

We're looking for a declarative approach as a replacement of Microsoft.Identity provider to provision AAD applications (add certificate, configure SN+I, etc.) on all tenants (xME and non-ME).

dkershaw10 commented 6 months ago

@huajunzhao let's continue this conversation over email.

dkershaw10 commented 6 months ago

Is this bicep template going to be available for Azure AD B2C ? To create app registrations there for web authentication using MSAL.

@Keilor2019 app registration is part of the private preview offering, although I think we are pivoting towards the newer Entra External ID for customers. While app config is supported, some elements around External ID apps is not yet, like Bicep types for user flows, MFA/CA policy, tenant branding, etc.

@Bandgren @frankz217 @kdambekalns - we are still on track for public preview in May. Sorry we can't deliver this sooner (I wish we could). @Bandgren - interested to learn more about your super nice use case.

weikanglim commented 5 months ago

@alex-frankel @dkershaw10

There's been some customer interest around using the Graph provider for multitenant scenarios, stemming from some observations that some organizations prefer to have app registrations done in a separate tenant from the tenant that houses their Azure resources:

  1. Suppose I wanted to create an app in tenant foo, would I end up doing the following? 
    az login --tenant-id foo
az deployment tenant create --location westus --template-file app.bicep # Or does it matter what scope I use for the deployment here?
  2. Suppose I wanted to reference an app in tenant foo, but I'm in the context of a bicep deployment with tenant bar -- is this possible?
franklixuefei commented 4 months ago

@alex-frankel

Is Graph provider available now for internal use? If yes, could you please share the guidance?

We're looking for a declarative approach as a replacement of Microsoft.Identity provider to provision AAD applications (add certificate, configure SN+I, etc.) on all tenants (xME and non-ME).

Hi @huajunzhao , just purely out of curiosity, what's wrong with using the Microsoft.Identity Provider? that's what our team has been using. I'd love to learn about your experience with it. Thanks!

veikkoeeva commented 4 months ago

@franklixuefei About using managed identities...

Would it be possible to create a service principal and a federated identity using Bicep just under tenant and then assign this federated identity to GitHub agent to deploy the high level structures from Bicep? I am thinking a mimimal script by a person and then bootstrapping e.g. a root management group just under tenant and so on. It looks to me this requires scripting like:

$appRegistrationName = "root-deployer"
$AppJson = az ad app create --display-name $appRegistrationName --query "{appId: appId, id: id}" --output json
$AppData = $AppJson | ConvertFrom-Json
$AppId = $AppData.appId
$AppObjectId = $AppData.id

# Create the Service Principal
$SpJson = az ad sp create --id $AppId --query "{id: id}" --output json
$SpData = $SpJson | ConvertFrom-Json
$SpAssigneeObjectId = $SpData.id

$CredentialName = $appRegistrationName
$Request = @{
    name = $CredentialName
    issuer = "https://token.actions.githubusercontent.com"
    subject = "repo:${GitHubOwner}/${GitHubRepo}:ref:refs/heads/${GitHubRepoBranch}"
    description = "Federate GitHub Action"
    audiences = @("api://AzureADTokenExchange")
}

$RequestJson = ($Request | ConvertTo-Json -Compress) -replace '"', '\"'

(assign roles here)

az rest --method POST --uri "https://graph.microsoft.com/beta/applications/${AppObjectId}/federatedIdentityCredentials" --headers "Content-Type=application/json" --body $RequestJson

dazinator commented 4 months ago

Automating deployments with bicep is fine, but I think the struggle is establishing a suitable service principal and RBAC permissions for it to allow the automated bicep deployment to be setup for unattended scenarios and to succeed.

They are two different use cases (establishing a suitable service principal, and the bicep deployment itself) , but I think people want to streamline it as much as possible and look to use bicep to establish both sides.

I feel Azure / ARM / Bicep should make this a lot easier.

For example, following the idea of granting the least amount of permissions needed to accomplish the bicep deployment - bicep and ARM are actually in the best position to inspect the Bicep / ARM deployment and then tell me what are the permissions that need to be granted, to install this thing.

I can imagine multiple different solutions to streamline this. One idea would be:

  1. Bicep can generate a manifest of the required permissions needed to run the deployment
  2. A new experience for adding a bicep deployment to a tenant. Similar to open id connect scopes confirmation, an authenticate azure user can "add" the bicep deployment to their tenant, making use of the permissions manifest - this would somehow establish a service principal for the deployment with the required permissions. This service principal can now be used to run the bicep deployment in unattended scenarios.
  3. If a user attempts to run the bicep deployment and this manifest is present, rather than fail half way in the deployment when a user doesnt have some permission for something, the manifest can be checked right at the start and the deployment rejected stating the current user doesnt have permissions Xyz.
  4. Bicep deployments change and new permissions might be required over time. Some way to "refresh" the established service principal for unattended deployments so it is up to date with the current manifest.

We are just recognising here that Bicep deployment requires an authenticated sessions, and requires the user to be authorised to deploy its resources. Rather than authZ checks happening mid way through a deployment they should ideally be first before any change is made. This is similar to best practice in any API request or CLI command in applications today -authentication and authorisation are done early. Second, if you have a human manually running the bicep deployment in an attended fashion, you are probably doing things wrong. Realistically people want pipelines, and being able to establish one needs to be much easier.

dkershaw10 commented 4 months ago

Hi @dazinator - you are spot on with your assertions. We've been talking internally about rather than whatif just failing (if the calling principal doesn't have the requisite permissions), that it could tell what permissions are missing - or simply run in a mode that inspects the entire file and tells you what permissions are required. NOTE: as the Bicep team introduces extensibility, these required permissions span Azure resources and the extensible resources, such as Microsoft Graph resources like Entra ID.

This isn't something that is currently supported, but it is on our radar, whether through whatif or another mechanism.

The idea you have for #2 is something that can be explored. Clearly the user granting/assigning permissions to the service principal needs to have the requisite privileges/role.

@veikkoeeva - the script you have can be done through Bicep - we just blogged an announcement for the release tomorrow - here: https://techcommunity.microsoft.com/t5/azure-governance-and-management/announcing-public-preview-of-bicep-templates-support-for/ba-p/4141772.

dkershaw10 commented 4 months ago

@alex-frankel @dkershaw10

There's been some customer interest around using the Graph provider for multitenant scenarios, stemming from some observations that some organizations prefer to have app registrations done in a separate tenant from the tenant that houses their Azure resources:

  1. Suppose I wanted to create an app in tenant foo, would I end up doing the following?
az login --tenant-id foo
az deployment tenant create --location westus --template-file app.bicep # Or does it matter what scope I use for the deployment here?
  1. Suppose I wanted to reference an app in tenant foo, but I'm in the context of a bicep deployment with tenant bar -- is this possible?

@weikanglim - sorry for the late response. In theory this should be possible, but it would require separate template deployments, since AFAIK a template's maximum deployment scope is at a single tenant level. Your first deployment (1) deploys the multi-tenant app in tenant1. The second deployment in tenant2 deploys a service principal (using the appId for the app in tenant1) - and then that can be used/referenced in any Azure resources that may have a dependency on that app.

The nasty bit is that you need to feed the output from template1 (appId) as a parameter to template2, and that you would need to deploy these templates using different security principals.

Not sure if there are some better ways to do this...

veikkoeeva commented 4 months ago

@dkershaw10

@veikkoeeva - the script you have can be done through Bicep - we just blogged an announcement for the release tomorrow - here: https://techcommunity.microsoft.com/t5/azure-governance-and-management/announcing-public-preview-of-bicep-templates-support-for/ba-p/4141772.

You mean both creating the application and the service principal or another one of them? I might be a slow, but couldn't decipher the blog post how could I do that creation of service principal an assigning it to the app and then giving that to e.g. GitHub to deploy management groups, subscriptions etc. to a tenant using Bicep.

dkershaw10 commented 4 months ago

@veikkoeeva I assume that you are talking about creating a GitHub action to push a Bicep file deployment to the Azure infrastructure tenant (tenant2), and that you want to set this up using federated identity credentials?

The public docs and samples are released tomorrow, and there's an example of configuring federated identity credentials for GitHub actions that can access/deploy Azure resources. However the example doesn't split things across tenants, and I've not tried that before. But in theory, it should be possible:

Deploy to tenant 1 - the Identity tenant - create app reg included FIC for GitHub

provider microsoftGraph

@description('Subject of the GitHub Actions workflow\'s federated identity credentials (FIC) that is checked before issuing an Entra ID access token to access Azure resources. GitHub Actions subject examples can be found in https://docs.github.com/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect#example-subject-claims')
param githubActionsFicSubject string

var githubOIDCProvider = 'https://token.actions.githubusercontent.com'
var microsoftEntraAudience = 'api://AzureADTokenExchange'

// create a multi-tenant application reg, with FIC
resource githubActionsApp 'Microsoft.Graph/applications@v1.0' = {
  uniqueName: 'githubActionsApp'
  displayName: 'Github Actions App'
  signInAudience: 'AzureADMultipleOrgs' // make this app multi-tenant

  resource githubFic 'federatedIdentityCredentials' = {
    name: '${githubActionsApp.uniqueName}/githubFic'
    audiences: [microsoftEntraAudience]
    description: 'FIC for Github Actions to access Entra protected resources'
    issuer: githubOIDCProvider
    subject: githubActionsFicSubject
  }
}

output multiTenantAppId string = githubActionsApp.appId

Deploy to tenant 2 - the Azure infrastructure tenant - create SP and assign it necessary roles

provider microsoftGraph

@description('Role definition ID to be assigned to allow the Bicep file deployment by GitHub action')
param roleDefinitionId string = 'b24988ac-6180-42a0-ab88-20f7382dd24c' // Contributor

// appID output from the previous Bicep deployment in tenant1
@description('AppID of the multi-tenant app')
param multiTenantAppId string

// Create a service principal in tenant2 using the multi-tenant app registered in tenant 1
resource githubActionsSp 'Microsoft.Graph/servicePrincipals@v1.0' = {
  appId: multiTenantAppId
}

// The service principal needs to be assigned the necessary role to access the resources
// In this example, it is assigned with the `Contributor` role to the resource group
// which will allow GitHub actions to access Azure resources in the resource group via Az PS/CLI
var roleAssignmentName = guid('githubActions', roleDefinitionId, resourceGroup().id)
resource roleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: roleAssignmentName
  properties: {
    principalId: githubActionsSp.id
    principalType: 'ServicePrincipal'
    roleDefinitionId: resourceId('Microsoft.Authorization/roleDefinitions', roleDefinitionId)
  }
}

Hope this helps get the general idea across, assuming I've understood what you are trying to accomplish.

UPDATE: Our GitHub repo for Microsoft Graph Bicep is now public, so you can take a look at the quickstart sample we have there for configuring GitHub actions to gain access to Azure resources, using federated identity credentials.

dkershaw10 commented 4 months ago

Folks - we've announced availability of the public preview of Bicep templates for Microsoft Graph resources - initially for core Microsoft Entra ID resources, which should mostly address this issue.

If you find any issues with the Microsoft Graph Bicep extension or any missing features or missing Bicep types, please use the Microsoft Graph Bicep GitHub repo to file issues.

alex-frankel commented 4 months ago

🎉🎉🎉🎉 - very happy to close this one!

veikkoeeva commented 3 months ago

@veikkoeeva I assume that you are talking about creating a GitHub action to push a Bicep file deployment to the Azure infrastructure tenant (tenant2), and that you want to set this up using federated identity credentials? [...] UPDATE: Our GitHub repo for Microsoft Graph Bicep is now public, so you can take a look at the quickstart sample we have there for configuring GitHub actions to gain access to Azure resources, using federated identity credentials.

Thanks!

As an aside (and for lurkers here), it appears I can't az stack mg create since I get insufficient privileges for graph. However, e.g. az deployment mg works. This probably worth noting since I liked, initially, to create a new "company root management group" under Tenant Root Group using a Bicep, assign the principal to that, give it to the GH repo that then deploys the rest under this "company root management group". Probably preferably as stack.

<Edit: It may be the underlying reason is

A management group-scoped stack is restricted from deploying to another management group. It can only deploy to the management group of the stack itself or to a child subscription.

from https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/deployment-stacks?tabs=azure-powershell#known-issues