jason-johnson / azure-pipelines-tasks-terraform

Azure Pipelines extension for Terraform
MIT License
127 stars 55 forks source link

multi-stage pipeline publishPlanResults #268

Closed LRR1993 closed 1 year ago

LRR1993 commented 2 years ago

Describe the bug When using a multi-stage pipeline, which has more than 1 terraform plan run, when using publishPlanResults (all with different names), no terraform plan results are published. I have run a single stage and it works fine

To Reproduce Steps to reproduce the behavior:

  1. Setup pipeline as (include yaml configuration or screenshots of classic ui editor)
stage: BaseInfrastructureUpdates
    jobs:
       - job:
         steps:
              - task: TerraformCLI@0
                 displayName: ${{ parameters.displayName }}
                 inputs:
                     environmentServiceName: **
                     providerServiceAws: **
                     providerAwsRegion: eu-west-1
                     backendType: aws
                     backendServiceAws: **
                     publishPlanResults: ${{ parameters.publishPlanResults }}
stage: Microservices
    jobs:
       - job:
         steps:
              - task: TerraformCLI@0
                 displayName: ${{ parameters.displayName }}
                 inputs:
                     environmentServiceName: **
                     providerServiceAws: **
                     providerAwsRegion: eu-west-1
                     backendType: aws
                     backendServiceAws: **
                     publishPlanResults: ${{ parameters.publishPlanResults }}
  1. Execute pipeline
  2. No plans results

Expected behavior All terraform plans to be shown

Screenshots Screenshot 2022-04-07 at 08 51 15

Agent Configuration

bothzoli commented 2 years ago

At least I'm not the only one with this issue πŸ˜„ (see #267)

CorrenSoft commented 2 years ago

In your example, you are not including command: plan. Could be that?

I have been using this task in multi-stage pipelines for a while and I have never faced this issue, with or without empty plans.

lerlacher-haylix commented 2 years ago

Commenting here to +1 this and add some more debugging info.

This is the error I see in the console:

image

Interestingly, it only shows up once - for what I think is the first plan that was run out of 8. So the 404 is causing the view to barf without trying to process any other plans.

As a test, I added some conditionals to my pipeline definition to only add the publishPlanResults input to the runs where I expected changes. With that, the plan that has changes is showing up in the plan view.

So this also strongly points in the direction of an issue with empty plans.

lerlacher-haylix commented 2 years ago

So it took me a while but this issue is actually really simple.

This is the regex for an empty plan:

https://github.com/jason-johnson/azure-pipelines-tasks-terraform/blob/2328fcd61e3119b7e5cf7b98c69608ad4c6ca700/tasks/terraform-cli/src/commands/tf-plan.ts#L12

This is the actual output from Terraform:

image

No changes. Your infrastructure matches the configuration.

From the debug log:

2022-08-23T01:18:17.4756647Z No changes. Your infrastructure matches the configuration.

So the simple fix is to update the regex, maybe just to /^No changes./. A better fix may be to remove the parsing entirely from this case:

https://github.com/jason-johnson/azure-pipelines-tasks-terraform/blob/2328fcd61e3119b7e5cf7b98c69608ad4c6ca700/tasks/terraform-cli/src/commands/tf-plan.ts#L56-L60

If there are no changes, then the output doesn't need to be parsed, a simple static "No changes" message can be output instead.

@jason-johnson I don't have any development environment for azure devops extensions and I won't be able to set one up on short notice, is there any way you could fix up the regex and publish a preview version or something?

jason-johnson commented 2 years ago

I'm still getting my environment set up from the switch over and don't yet have an environment either.

jmyers82 commented 2 years ago

@LRR1993 First there has to be a change in the plan. If there is no change it wont show anything. I would like to see that functionality changes to show the plan REGARDLESS.

So in my case I am giving the publishPlanResults a unique file name for each environment - $(Build.DefinitionName)-$(Build.BuildId)-${{ parameters.env }}.tfplan

Then in the pipeline gui I get the option to pick which one I want to see: image

Possibly what is happening on yours is that it is over writting it?

vanwx commented 2 years ago

@jmyers82

I tried to add a buildId to the plan output name as well. It still the same issue. image

jmyers82 commented 2 years ago

@vanwx not sure that it matters but can you make the file name end with .log Not sure that is the actual problem in your case but that is the one difference I see. Can you put in the section of code where you make the call for the plan?

vanwx commented 2 years ago

@jmyers82

Same with the .log. I noticed the first requests to pull all possible plan results succeeded with good response including all the plans. But the request to get the individual plan result failed with 404 code image

lerlacher-haylix commented 2 years ago

I think this is all the same underlying bug as I noted up-thread https://github.com/jason-johnson/azure-pipelines-tasks-terraform/issues/268#issuecomment-1223412278. The regex for detecting "No changes" is broken and the correct code path for dealing with an empty plan is never taken.

vanwx commented 2 years ago

@lerlacher-haylix I think you are probably right. The dropdown list probably pick the last plan result from the multi-stage build. That last one might not have any changes and were not pushed like you said hence the api resulted 404.

As a result the dropdown can't be rendered due to js error πŸ˜…

jmyers82 commented 2 years ago

Sorry @vanwx I made the assumption that there were infact changes in the plan. Yeah I dont like how the NO CHANGE process works. I would like it to render the plan showing no changes still yet.

vanwx commented 2 years ago

@jmyers82 no worries at all.

I can confidently confirm the issue about NO CHANGE process is the problem. It doesn't push log files or unchanging plans.

As my case above, the pipeline has only 1 stage that produces the plan changes. That stage is not the last stage which doesn't have any changes and were not pushed the plan logs.

The dropdown can't be rendered due to js error as it defaults to select the last pushed logs. When I manually copy the url for the changing plan and open in browser, I can indeed get the logs succesfully.

Fixing the regex seems simple, can we just send PR with above suggestions @jason-johnson ?

cedricfortin commented 1 year ago

I also suffer the same problem in a multi stage pipeline.

Thanks to @lerlacher-haylix comments and following @vanwx idea I've created a PR πŸ˜„

@jason-johnson can you please review it?

Thanks!

lerlacher-haylix commented 1 year ago

I'm extremely happy to see progress on this and other activity in the project.

Is there a path to getting a release cut?

Is there anything I can do to help? Happy to set up a dev environment to test this out but this is very far from what I normally work on so no idea where to start.