hashicorp / terraform-provider-azurerm

Terraform provider for Azure Resource Manager
https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs
Mozilla Public License 2.0
4.6k stars 4.64k forks source link

Logic App resource definitions don't make sense #5197

Closed ghost closed 4 years ago

ghost commented 4 years ago

This issue was originally opened by @brandonh-msft as hashicorp/terraform#23694. It was migrated here as a result of the provider split. The original body of the issue is below.


Terraform Version

0.12.18

Expected Behavior

Logic App definitions should be completely within one resource of Terraform. When you're defining a Logic App on Azure, you're doing so with a single ARM template, not separate templates for the trigger and every action inside the Logic App.

Actual Behavior

This decomposition w/in Terraform is confusing and inaccurate, especially when you're looking at looping, parallel branches, etc of a Logic App.

rrmistry commented 4 years ago

We use VS Code with Logic App extension.

This makes it easy to edit / debug / copy-paste JSON definition into terraform ARM template 1-to-1 vsCodeLogicApp1 vsCodeLogicApp2 vsCodeLogicApp3

Currently the terraform resource style definition of actions and triggers is not compatible with other parts of Logic App ecosystem.

Possible solution to preserve backward compatibility: add definition as an object argument to azurerm_logic_app_workflow

Would be helpful to convert parameters argument into object type instead of map type to support arbitrary API $connections (#1691)

drdamour commented 4 years ago

Currently the terraform resource style definition of actions and triggers is not compatible with other parts of Logic App ecosystem.

i'm curious for concrete examples @rrmistry as we're just starting to get into this. Not a big fan of arm because of how it can't detect a change made through the portal

rrmistry commented 4 years ago

Not a big fan of arm because of how it can't detect a change made through the portal

+1 ^

Simplest approach:

Drawbacks of simplest approach:

My approach:

Break definition into 3 files:

  1. azure-logicapps.tf To house the logic app terraform ARM template resource
  2. logicapp.azrm.json To house the ARM template definition
  3. definition.logicapp.json To house the Logic App definition

Design decisions:

(not proud of amount of hacking needed to move project forward)

At the moment, cannot avoid drift between Azure Logic App definition and local terraform definition

As a workaround we limit Azure deployed Logic Apps to read-only (not ideal)

My ideal case would be to have logic app definitions/parameters be stored to json, parameterized with terraform variables

The azurerm provider could then call Logic App REST API to get current "cloud" definition and compare with local definitions/parameters to evaluate if a re-deployment is necessary (after terraform variable substitution is complete)

Possible concerns: https://github.com/terraform-providers/terraform-provider-azurerm/commit/1079afa4a9889304cce16d9807d8f22b56309a04#r36216441 terraform api makes everything lower case so doing a diff may be difficult

tombuildsstuff commented 4 years ago

👋🏻 hey folks

Before diving into this, I think it's probably worth explaining why the resources are designed the way they are.

Logic Apps have primarily been designed with the expectation they'll be managed in the GUI inside the Azure Portal. Whilst they do support being managed through the ARM API's (and Templates) - this arguably isn't the primary interaction model - and the API's aren't really designed with this model in mind.

Ultimately this means the integration options for this are essentially submitting a giant JSON blob and hoping it works, which whilst can work in some circumstances ends up causing other issues around chaining dependencies (either during creation, or destroy). In order to correctly map dependencies you'd need to first apply one workflow, then configure the dependencies, then re-configure the workflow to include the dependencies.

Whilst that model works for ARM Templates, since most folks (in my experience) are wrapping the Template Deployment with either a Bash/PowerShell script anyway - this isn't something that works in Terraform, since this'd lead to either conflicting/circular reference between resources.


For comparison - Data Factory has a similar scope to Logic Apps insofar as it supports interacting/plugging into a large number of external sources; however the key difference between the two is the way the API's designed, insofar as there's endpoints to add new Data Sets/Linked Services to this.

What this means is that it's possible to create composable resources together, for example creating a Data Factory, then the Database and then link the services as required in a single run.


Due to the way the Logic Apps API is currently designed, unfortunately adding a field for workflow won't work in Terraform, since users will either end up with circular references between resources - or be unable to detect drift between these resources. As such unfortunately at this point in time we're forced to implement these as virtual resources to be able to correctly represent dependencies between resources (and track diff's).

Whilst I appreciate this does diverge from the Azure API - to be able to correctly represent Logic Apps in Terraform we need to manage the workflow field via these Virtual Resources.

Ultimately this comes down to the design of the Logic Apps API being unsuited to Terraform - where the Logic Apps API needs endpoints for managing connections, parameters and actions (amongst other things). As such I'd suggest the best path forward here would be to work with the Service Team to expose the necessary API's to fully support Logic Apps as Code going forward (which'll aid everybody integrating with Logic Apps, rather than just Terraform).

However for the reasons outlined above unfortunately we have no plans to expose the workflow field in the azurerm_logic_app_workflow and as such I'm going to close this issue for the moment.

Thanks!

drdamour commented 4 years ago

@tombuildsstuff I think you're underestimating how many logic apps are just used as glue to pre-existing apis. These are at the very top and nothing is depending on them. Mostly these are constructed in the gui, and json is just copied out to terraform to promote up the dev->preprod*->prod deployment chain. Would those still suffer from this circular dependency youre concerned of?

brandonh-msft commented 4 years ago

these are constructed in the gui, and json is just copied out to terraform to promote up the dev->preprod*->prod deployment chain.

literally dozens of enterprise customers doing that exact workflow. Even still today VSCode provides pseudo-offline editing and committing directly to SCC.

@tombuildsstuff all due respect but if this is the case you should remove all logic apps resources from TF - the ones that exist today simply are not useful - and put a note in the AzureRM docs that LA is not and will never be supported so people don't chase their tail and keep filing issues on you.

drdamour commented 4 years ago

@tombuildsstuff would u be open to a PR that allowed workflow to take a heredoc of the whole logic app workflow as a "simple mode" option, and make it an either or like some of the other resources are configured throughout TF?

tombuildsstuff commented 4 years ago

@drdamour unfortunately even if we exposed that field we'd end up with the majority of users having circular reference issues here, which'd cause more problems than it solved - so unfortunately we wouldn't at this time.

As mentioned above unfortunately I don't see a way that we can do much for Logic Apps until the Logic Apps API is more composable - as such I'm going to lock this issue for the moment - but when the Logic Apps API becomes more composable we can take another look.

Thanks!