Documentation example enchancement request: API Management revision handling

stutommi commented 2 years ago

Is your feature request related to a problem? Please describe. I have a problem understanding how to deal with API Management revisions using bicep. There is a great example about API Management deployment in examples, but it doesn't really address revision handling in anyway.

When I try it using for example the following block:

resource api 'Microsoft.ApiManagement/service/apis@2020-12-01' = {
   ...
  properties: {
    ...
    apiRevision: 'revisionName' // <------ Revision specific property
    apiRevisionDescription: 'This is revision text' // <------ Revision specific property
    isCurrent: false // <----- Revision specific property
    ...
  }
}

properties apiRevision and isCurrent seem to be ignored, and I'll always end up with no revision (= default revision "1" is created).

Describe the solution you'd like I would like to see how bicep can be used to handle creation of new revisions.

alex-frankel commented 2 years ago

@brwilkinson -- do you happen to have any context on this one?

brwilkinson commented 2 years ago

Sure, I'll take a look.

brwilkinson commented 2 years ago

Okay I did post the response; however I will do some more testing first and then re-update the thread.

brwilkinson commented 2 years ago

given I posted the response, I figured I would add back in these comments, until I can provide an update.

Hi @stutommi

A few recommendations before we get started.

1) the APIM extension for vscode is really useful for managing/testing api's 2) I would also take a look at the git feature to get familar with pulling down the code from APIM as well. 1) https://docs.microsoft.com/en-us/azure/api-management/api-management-configuration-repository-git

I think we can make it work in Bicep, however I am skeptical about how practical this is based on other tools such as the VSCode extension or even the the Developer Portal for managing/Testing the API's, plus like I said I recommend using the Git feature. I guess it just depends on your use case.

So I was able to clone these to new revisions (as seen below), however I realized that I had not cloned all of the related Operations. that go along with them.

Let me take another look at that part. I can definitely clone them all individually by name, however I think this process only make sense if I can list them all for a given API and then clone them that way. Without having to maintain a list of each operation e.g. GET/SET etc etc. for each api operation.

brwilkinson commented 2 years ago

Okay I made the update.... I figured I would just start with passing in the operation names, since that is enough to get a feel if this would even be useful for you to test.

This is what you pass in...

the revision that you want to clone
the revision that you want to clone it to, it can be a new or existing revision.
The operation-id list that the api supports.

APIM-API.bicep

Sample link: https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/APIM-API.bicep

param apim object = {
  name: 'ACU1-BRW-AOA-T5-apim01'
}
param apis array = [
  {
    name: 'echo-api'
    clonefrom: '1'
    cloneto: '5'
    addrevisiondescriptionprefix: 'test new revision'
    Operations: [
      'create-resource'
      'modify-resource'
      'remove-resource'
      'retrieve-header-only'
      'retrieve-resource'
      'retrieve-resource-cached'
    ]
  }
]

module getApiCurrent 'APIM-API-Get.bicep' = [for (api, index) in apis: {
  name: 'dpGetAPI-CurrentRev-${api.name}'
  params: {
    apim: apim
    api: api
  }
}]

module setNewRevision 'APIM-API-Clone.bicep' = [for (api, index) in apis: {
  name: 'dpCreateClone-${api.name}-rev${api.cloneto}'
  params: {
    apim: apim
    api: getApiCurrent[index].outputs.currentapi.properties
    operations: getApiCurrent[index].outputs.currentapioperations
    apinew: api
    operationNames: api.Operations
  }
}]

output current array = [for (api, index) in apis: getApiCurrent[index]]
output newrevision array = [for (api, index) in apis: setNewRevision[index]]

APIM-API-Get.bicep

Sample link: https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/APIM-API-Get.bicep

This now gets:

The current revision
The current operations

param api object
param apim object

var revisionName = api.clonefrom == '1' ? api.name : '${api.name};rev=${api.clonefrom}'

resource APIM 'Microsoft.ApiManagement/service@2021-04-01-preview' existing = {
  name: apim.name
}

resource API 'Microsoft.ApiManagement/service/apis@2021-04-01-preview' existing = {
  name: revisionName
  parent: APIM
}

resource APIOperations 'Microsoft.ApiManagement/service/apis/operations@2021-04-01-preview' existing = [for (op, index) in api.Operations : {
  name: op
  parent: API
}]

output currentapi object = API
output currentapioperations array = [for (item, index) in api.Operations : APIOperations[index]]

APIM-API-Clone.bicep

Sample link: https://github.com/brwilkinson/AzureDeploymentFramework/blob/main/ADF/bicep/APIM-API-Clone.bicep

This now clones both of the:

The new revision
The operations

param apinew object
param api object
param apim object
param operations array
param operationNames array

resource APIM 'Microsoft.ApiManagement/service@2021-04-01-preview' existing = {
  name: apim.name
}

resource API 'Microsoft.ApiManagement/service/apis@2021-04-01-preview' = {
  name: '${apinew.name};rev=${apinew.cloneto}'
  parent: APIM
  properties: {
    displayName: api.displayName
    apiRevision: apinew.cloneto
    subscriptionRequired: api.subscriptionrequired
    serviceUrl: api.serviceUrl
    path: api.path
    protocols: api.protocols
    apiRevisionDescription: '${apinew.addrevisiondescriptionprefix} ${apinew.cloneto} from: rev=${apinew.clonefrom}'
  }
}

resource APIOperations 'Microsoft.ApiManagement/service/apis/operations@2021-04-01-preview' = [for (op, index) in operations : {
  name: operationNames[index]
  parent: API
  properties: op.properties
}]

output currentapi object = API

It would be nice if I could just use the API Name and revision to lookup the Operations, then clone them, without having to manage that list of operations. I will have a look into that.

I am happy with this solution, I clone down ALL of the APIM configuration locally via the source control option and I have tested with diffs on the clone from and clone to and I am happy this should work well. However, let me know how it goes for you.

stutommi commented 2 years ago

Hi @brwilkinson!

Thanks for the great effort on making the example, there a lot new stuff for me to digest. For the practical application, I think it would be good for me to describe the kind of revision handling I'm most interested in.

Let's take a following scenario:

APIM configuration doesn't need to be managed. We can assume that the APIM is always going to be present.
APIM API related bicep is being run only Azure devops pipeline as part of CI/CD.
APIM API Policy (xml-file) and openapi definition (YAML-file) live in git version control.
APIM API infra files live together in the same repository as the API source code itself. The openapi spec gets generated by the API endpoints (or might be manually created/edited).

I would want the APIM API deployment happen, whenever either API policy, openapi or any files regarding API deployment (=bicep files) change. Whenever this happens, I would like a new revision to be generated for API in Api management, and most preferable be instantly promoted to current revision. The older revision would stick around, allowing possible manual rollback (handled from Azure portal).

The main thing would be, There would always be new revision after CI/CD deployment (if necessary files have changed) set as new current, and previous revision would be still be around for rollback.

Can you figure out any way of making this happen easily with Bicep? I know this can be handled with Powershell scripting, but I'm trying to find a way of implementing it with bicep. Would you think this kind of flow that I'm describing would be better handled with some other way than bicep?

brwilkinson commented 2 years ago

okay, that all sounds good.

One clarification, are you pushing/pulling any changes via the APIM integrated source control? Or was that in a different/external repo?

stutommi commented 2 years ago

No APIM integrated source control in use! I haven't stumbled upon it before, look interesting (and a totally different flow). The note on the docs looks a bit scary though:

This feature is designed to work with API Management services that have a small/medium configuration. Services with large number of configuration elements (APIs, Operations, Schemas etc.) may experience unexpected failures when processing Git commands. If you encounter such failures, please reduce the size of your service configuration and try again. Contact support if you need assistance.

brwilkinson commented 2 years ago

okay, thanks for confirming... in that case everything is push and you don't need to pull any changes from APIM back out to keep them in sync.

So I think the above solution may work for you, however instead of reading and cloning the previous revision and operations via the APIM-API-Get.bicep, you should already have enough information to populate those values in the new revision.

stutommi commented 2 years ago

Thanks for the help, great stuff! :)

Just a follow up: It's not possible to set newly created revision as current at the time of deployment, so I changed my flow a bit, seems to be working:

On every new deployment, update always present revision called "latest" (which is always set as current revision), and create a rollback revision (with an identifiable revision name tied to the CI/CD deployment) that is not current. This enables rollback versions while having the newest deployment revision also as current one (in the latest-revision). Only problem I find with this is pollution of revisions if new api specs are deployed often. I have no idea how consuming it is for APIM-instance if every API gets filled with rollback revisions. You can of course always delete them manually, but that's one manual process one would like to avoid.

I'll leave this still open if you have any comments @brwilkinson, but I'm very happy with things you showed and found an answer to my own implementation.

Thanks alot!

brwilkinson commented 2 years ago

I'll move this over to the 'discussions' page for now, instead of just closing out the issue.

Update, I decided not to, since I don't think it will take the history over.

brwilkinson commented 2 years ago

Thanks @stutommi I will mark as resolved, glad you found a working solution for your needs.

Azure / bicep