search

Azure / apiops

APIOps applies the concepts of GitOps and DevOps to API deployment. By using practices from these two methodologies, APIOps can enable everyone involved in the lifecycle of API design, development, and deployment with self-service and automated tools to ensure the quality of the specifications and APIs that they’re building.
https://azure.github.io/apiops
MIT License
312 stars 182 forks source link

[Question] How to not include "Managed" gateway by default when publishing APIs #501

Open daviian opened 6 months ago

daviian commented 6 months ago

Release version

v5.1.1

Question Details

Our APIM solely uses self-hosted gateways and thus does not require every API to be assigned to the "Managed" gateway. However, even though the API is only assigned to a specific self-hosted gateway, the "Managed" gateway is still getting assigned to it.

I added a "Managed" gateway into the gateways directory and set the APIs to an empty array. However, this only works when not using the commit id during publishing, as the Managed gateway file is not changed when I add a new API.

How is this supposed to be handled?

Expected behavior

"Managed" gateway is not assigned whenever I assign it to some custom gateways.

Actual behavior

"Managed" gateway is always assigned to new APIs.

Reproduction Steps

Create a fresh API and assign it to a custom gateway (not the "Managed" one)

Folder structure: image

Content of gateways/self-hosted/apis.json

[
  {
    "name": "my-new-api"
  }
]

Content of gateways/Managed/apis.json

[]

Since the gateways/Managed/apis.json has not been touched/changed during the commit, the publisher does not update the Managed gateway assignment, so the new API is actually having both the "Managed" and "self-hosted" gateways assigned to it.

github-actions[bot] commented 6 months ago 
  Thank you for opening this issue! Please be patient while we will look into it and get back to you as this is an open source project. In the meantime make sure you take a look at the [closed issues](https://github.com/Azure/apiops/issues?q=is%3Aissue+is%3Aclosed) in case your question has already been answered. Don't forget to provide any additional information if needed (e.g. scrubbed logs, detailed feature requests,etc.).
  Whenever it's feasible, please don't hesitate to send a Pull Request (PR) our way. We'd greatly appreciate it, and we'll gladly assess and incorporate your changes.
guythetechie commented 6 months ago

There is a similar issue with products in #245. This new implementation will fix it, and we're adding a similar one for gateways. Should be in the next major release.

daviian commented 6 months ago

I have my doubts that the proposed changes actually resolve this issue. Let me share my train of thought and feel free to correct me if I am wrong. Every addition of an API contains a changed file in the self-hosted gateway folder, so this is all good. The publisher considers them during publishing of the git commit. However, since I don't want to have the new API associated with the existing managed gateway, there are no file changes in the managed gateway folder, so the publisher is still not going to consider them. How is the proposed change addressing this?

guythetechie commented 6 months ago

I'm not sure what your baseline is, so I'll make some assumptions and walk through a few scenarios.

Let's say you start with a managed and self-hosted gateway in the portal. You have zero APIs. You run the extractor. Your folder structure will be:

You then go to the Azure portal, create APIs A and B, assign API A to the managed gateway, and API B to the self-hosted gateway. You run the extractor. Your folder structure will be:

You then decide to do a code-first deployment: instead of making changes via the portal & extracting, you want to change the git artifacts and publish that commit. In this change, you remove API A from the managed gateway, add a new API C to the self-hosted gateway, and add a new API D that won't be associated with any gateways. Your folder should look like this:

The publisher will detect the following changes and make them accordingly when it runs:

Does that make sense? Let me know if I'm missing something.

guythetechie commented 6 months ago

Something just clicked for me; I misread your initial question. When you talk about the "managed" gateway, you're not referring to a gateway you create called "managed". You're referring to the built-in managed gateway.

If that's the case, I doubt there's functionality to manage that "managed" gateway. You can't export it as an artifact and "manage" it. You can assign APIs to a self-hosted gateway using the methods described in the previous comment, but I'm not aware of a way to also disassociate them explicitly from the built-in gateway.

I would suggest creating a support ticket and asking whether that's possible (outside of our tools). If they show you how to do it, we can see if the steps are achievable through our tools.

daviian commented 6 months ago

Yes, I am talking about the built-in managed gateway. Sorry for the missing clarity.

I actually experimented and simply created a folder called "Managed" and added an apis.json that has an empty array in it. When I use the publisher without the COMMIT_ID it actually disassociates the APIs from the built-in gateway. But as I said, this does not work when only looking at changes within a git commit.

guythetechie commented 6 months ago

Can you elaborate on what you're seeing? How do you know they're no longer associated with the built-in gateway?

daviian commented 6 months ago

You can see the assigned built-in gateway when you have a look into the API in the portal. The field "Gateways" either contains the "Managed" + your custom ones, or only the custom ones after I tested it.

image

guythetechie commented 6 months ago

Thanks for the information - makes sense now. I think our implementation will partially handle your scenario.

When the extractor runs, it queries APIM for a list of gateways. It then loops through all gateways and exports their associated APIs. The problem is that APIM's list of gateways does not include the managed gateway. We would have to add a step to explicitly ask for the managed gateway's APIs. The APIs would then show up in artifacts/gateways/managed/apis/....

Adding to our backlog for further investigation.

waelkdouh commented 6 months ago

@daviian feel to submit a PR and we can merge accordingly.

daviian commented 6 months ago

Thanks for the information - makes sense now. I think our implementation will partially handle your scenario.

When the extractor runs, it queries APIM for a list of gateways. It then loops through all gateways and exports their associated APIs. The problem is that APIM's list of gateways does not include the managed gateway. We would have to add a step to explicitly ask for the managed gateway's APIs. The APIs would then show up in artifacts/gateways/managed/apis/....

Adding to our backlog for further investigation.

I suppose this also needs to be done in the publisher? Otherwise the built-in managed gateway is always assigned automatically.

daviian commented 6 months ago

@guythetechie - I've had a look into it and locally it looks good so far. However, there is one point I have concerns about. The APIM API documentation Gateway Api - Create Or Update and Gateway Api - List By Service excludes "managed" from being an allowed value ("Must not have value 'managed'"). However, both APIs work as intended for our use case when passing "managed". Additionally this API is also used from the APIM Portal when assigning/removing APIs from the managed gateway, so I assume the API documentation requires an update. The only API that returns a validation error is Gateway - Get, so I conditionally exclude fetching the gateway details for the "managed" gateway.

What are your thoughts on this? If you like I can create the PR so that you can have a preliminary look as well.

EDIT: There has been a similar question https://learn.microsoft.com/en-us/answers/questions/624766/azure-apim-gateways-and-bicep that has not been answered since quite a while.

daviian commented 5 months ago

@guythetechie @waelkdouh Any update? Anything I can do/provide to help out and create some progress?