jfrog / terraform-provider-xray

Terraform provider to manage JFrog Xray
https://jfrog.com/xray/
Apache License 2.0
149 stars 12 forks source link

plugin crash everytime on tf plan when upgrading to version >= 1.15.0 #141

Closed malamin closed 11 months ago

malamin commented 11 months ago

Describe the bug Terraform plan crashes after upgrade from 1.14.2 to any other higher versions (tested with v.2.0.0 incl.)

Requirements for and issue code snippet: image

Expected behavior Works still after the upgrade :)

Additional context image

alexhung commented 11 months ago

@malamin I'm unable to reproduce this crash yet. The error message shows the crash is caused by resource xray_repository_config but your HCL doesn't contain that.

This is the HCL I'm using with Terraform CLI 1.5.6 on a JFrog instance (Artifactory 7.69.4, Xray 3.82.10) that does not have Advanced Security.

terraform {
  required_providers {
    xray = {
      source  = "jfrog/xray"
      version = "1.15.0"
    }
  }
}

provider "xray" {
}

resource "random_id" "randid" {
  byte_length = 2
}

resource "xray_security_policy" "security1" {
  name        = "test-security-policy-severity-${random_id.randid.dec}"
  description = "Security policy description"
  type        = "security"

  rule {
    name     = "rule-name-severity"
    priority = 1

    criteria {
      min_severity          = "High"
      fix_version_dependant = false
    }

    actions {
      webhooks                           = []
      mails                              = ["test@email.com"]
      block_release_bundle_distribution  = true
      fail_build                         = true
      notify_watch_recipients            = true
      notify_deployer                    = true
      create_ticket_enabled              = false // set to true only if Jira integration is enabled
      build_failure_grace_period_in_days = 5     // use only if fail_build is enabled

      block_download {
        unscanned = true
        active    = true
      }
    }
  }
}

resource "xray_security_policy" "security2" {
  name        = "test-security-policy-cvss-${random_id.randid.dec}"
  description = "Security policy description"
  type        = "security"

  rule {
    name     = "rule-name-cvss"
    priority = 1

    criteria {

      cvss_range {
        from = 1.5
        to   = 5.3
      }
    }

    actions {
      webhooks                           = []
      mails                              = ["test@email.com"]
      block_release_bundle_distribution  = true
      fail_build                         = true
      notify_watch_recipients            = true
      notify_deployer                    = true
      create_ticket_enabled              = false // set to true only if Jira integration is enabled
      build_failure_grace_period_in_days = 5     // use only if fail_build is enabled

      block_download {
        unscanned = true
        active    = true
      }
    }
  }
}

resource "xray_license_policy" "license1" {
  name        = "test-license-policy-allowed-${random_id.randid.dec}"
  description = "License policy, allow certain licenses"
  type        = "license"

  rule {
    name     = "License_rule"
    priority = 1

    criteria {
      allowed_licenses         = ["Apache-1.0", "Apache-2.0"]
      allow_unknown            = false
      multi_license_permissive = true
    }

    actions {
      webhooks                           = []
      mails                              = ["test@email.com"]
      block_release_bundle_distribution  = false
      fail_build                         = true
      notify_watch_recipients            = true
      notify_deployer                    = true
      create_ticket_enabled              = false // set to true only if Jira integration is enabled
      custom_severity                    = "High"
      build_failure_grace_period_in_days = 5 // use only if fail_build is enabled

      block_download {
        unscanned = true
        active    = true
      }
    }
  }
}

resource "xray_license_policy" "license2" {
  name        = "test-license-policy-banned-${random_id.randid.dec}"
  description = "License policy, block certain licenses"
  type        = "license"

  rule {
    name     = "License_rule"
    priority = 1

    criteria {
      banned_licenses          = ["Apache-1.1", "APAFML"]
      allow_unknown            = false
      multi_license_permissive = false
    }

    actions {
      webhooks                           = []
      mails                              = ["test@email.com"]
      block_release_bundle_distribution  = false
      fail_build                         = true
      notify_watch_recipients            = true
      notify_deployer                    = true
      create_ticket_enabled              = false // set to true only if Jira integration is enabled
      custom_severity                    = "Medium"
      build_failure_grace_period_in_days = 5 // use only if fail_build is enabled

      block_download {
        unscanned = true
        active    = true
      }
    }
  }
}

resource "xray_watch" "all-repos" {
  name        = "all-repos-watch-${random_id.randid.dec}"
  description = "Watch for all repositories, matching the filter"
  active      = true

  watch_resource {
    type = "all-repos"

    filter {
      type  = "regex"
      value = ".*"
    }

    filter {
      type  = "package-type"
      value = "Docker"
    }
  }

  assigned_policy {
    name = xray_security_policy.security1.name
    type = "security"
  }

  assigned_policy {
    name = xray_license_policy.license1.name
    type = "license"
  }
  watch_recipients = ["test@email.com", "test1@email.com"]
}

resource "xray_repository_config" "xray-repo-config-pattern" {

  repo_name = "example-repo-local"

  paths_config {

    pattern {
      include             = "core/**"
      exclude             = "core/internal/**"
      index_new_artifacts = true
      retention_in_days   = 60
    }

    pattern {
      include             = "core/**"
      exclude             = "core/external/**"
      index_new_artifacts = true
      retention_in_days   = 45
    }

    all_other_artifacts {
      index_new_artifacts = true
      retention_in_days   = 60
    }
  }
}

resource "xray_repository_config" "xray-repo-config" {

  repo_name = "example-repo-local"

  config {
    vuln_contextual_analysis = true
    retention_in_days        = 90

    exposures {
        scanners_category {
          services     = true
          secrets      = true
          applications = true
        }
    }
  }
}
Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

random_id.randid: Creating...
random_id.randid: Creation complete after 0s [id=TNA]
xray_repository_config.xray-repo-config-pattern: Creating...
xray_repository_config.xray-repo-config: Creating...
xray_license_policy.license2: Creating...
xray_license_policy.license1: Creating...
xray_security_policy.security1: Creating...
xray_security_policy.security2: Creating...
xray_security_policy.security1: Creation complete after 0s [id=test-security-policy-severity-19664]
xray_license_policy.license2: Creation complete after 0s [id=test-license-policy-banned-19664]
xray_license_policy.license1: Creation complete after 0s [id=test-license-policy-allowed-19664]
xray_security_policy.security2: Creation complete after 0s [id=test-security-policy-cvss-19664]
xray_watch.all-repos: Creating...
xray_repository_config.xray-repo-config-pattern: Creation complete after 0s [id=example-repo-local]
xray_repository_config.xray-repo-config: Creation complete after 0s [id=example-repo-local]
xray_watch.all-repos: Creation complete after 1s [id=all-repos-watch-19664]
alexhung commented 11 months ago

@malamin I've also attempted creating the resources from last message using provider 1.14.2, upgrade provider to 1.15.0, then run terraform apply. No error and all the resources are updated correctly.

malamin commented 11 months ago

sorry, I forgot to paste that

resource "xray_repository_config" "xray_config" {

  repo_name = "example-repo-local"

  config {
    retention_in_days        = 90
  }
}

We have no advanced security so I guess that vuln_contextual_analysis should not be set then.

alexhung commented 11 months ago

@malamin Just re-test with the xray_repository_config configure you provided. No error when upgrading from 1.14.2 to 1.15.0, or from 1.14.2 to 2.0.0.

aserzhankou commented 11 months ago

Hello,

I also have problems with Xray provider upgrade from version to 1.12 to any version from 1.15 to 2.0.0.

In the plan I have a lot of UpgradeResourceState errors like this:

│ Error: Request cancelled
│ 
│   with module.remote-maven-spring-plugins.xray_repository_config.remote-maven-xray-config,
│   on .terraform/modules/remote-maven-spring-plugins/ops/terraform/modules/remote-maven-repo/main.tf line 48, in resource "xray_repository_config" "remote-maven-xray-config":
│   48: resource "xray_repository_config" "remote-maven-xray-config" {
│ 
│ The plugin.(*GRPCProvider).UpgradeResourceState request was cancelled.

And as a result:

Stack trace from the terraform-provider-xray_v2.0.0 plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xb03bcb]

goroutine 81 [running]:
github.com/jfrog/terraform-provider-xray/pkg/xray.resourceXrayRepositoryConfig.func7(0x0, {0xc000138778, 0x4})
        github.com/jfrog/terraform-provider-xray/pkg/xray/resource_xray_repository_config.go:354 +0x10b
github.com/jfrog/terraform-provider-xray/pkg/xray.resourceXrayRepositoryConfig.func8(0xc0007864e0, {0xc000451c70, 0x6}, {0xc000138778, 0x4})
        github.com/jfrog/terraform-provider-xray/pkg/xray/resource_xray_repository_config.go:381 +0x2b8
github.com/jfrog/terraform-provider-xray/pkg/xray.resourceXrayRepositoryConfig.func12({0x1c?, 0xcaae12?}, {{0xc0007b3660?, 0x203000?}, 0xc0007864e0?, 0x0?}, 0xc0003900c0?, {0xc000451c70, 0x6}, {0xc000138778, ...})
        github.com/jfrog/terraform-provider-xray/pkg/xray/resource_xray_repository_config.go:430 +0x117
github.com/jfrog/terraform-provider-xray/pkg/xray.resourceXrayRepositoryConfig.func14({0xddb5e8, 0xc000060b40}, 0xc00030e480, {0xc0b420?, 0xc000487740})
        github.com/jfrog/terraform-provider-xray/pkg/xray/resource_xray_repository_config.go:483 +0x64a
github.com/jfrog/terraform-provider-shared/util/sdk.applyTelemetry.func1({0xddb5e8?, 0xc000060b40}, 0x0?, {0xc0b420?, 0xc000487740})
        github.com/jfrog/terraform-provider-shared@v1.18.0/util/sdk/sdk.go:234 +0x263
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).read(0xc00049ec40, {0xddb620, 0xc000518180}, 0xd?, {0xc0b420, 0xc000487740})
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.27.0/helper/schema/resource.go:750 +0x12e
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).RefreshWithoutUpgrade(0xc00049ec40, {0xddb620, 0xc000518180}, 0xc0000b7040, {0xc0b420, 0xc000487740})
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.27.0/helper/schema/resource.go:1044 +0x59e
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ReadResource(0xc00030da58, {0xddb620?, 0xc000518030?}, 0xc0003907c0)
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.27.0/helper/schema/grpc_provider.go:616 +0x497
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).ReadResource(0xc0004b63c0, {0xddb620?, 0xc0002cea80?}, 0xc000176660)
        github.com/hashicorp/terraform-plugin-go@v0.18.0/tfprotov5/tf5server/server.go:751 +0x49e
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ReadResource_Handler({0xc60400?, 0xc0004b63c0}, {0xddb620, 0xc0002cea80}, 0xc000368070, 0x0)
        github.com/hashicorp/terraform-plugin-go@v0.18.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:386 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc00047c000, {0xdde2a0, 0xc0001024e0}, 0xc000355b00, 0xc000487560, 0x128ef90, 0x0)
        google.golang.org/grpc@v1.56.1/server.go:1337 +0xde3
google.golang.org/grpc.(*Server).handleStream(0xc00047c000, {0xdde2a0, 0xc0001024e0}, 0xc000355b00, 0x0)
        google.golang.org/grpc@v1.56.1/server.go:1714 +0xa1b
google.golang.org/grpc.(*Server).serveStreams.func1.1()
        google.golang.org/grpc@v1.56.1/server.go:959 +0x98
created by google.golang.org/grpc.(*Server).serveStreams.func1
        google.golang.org/grpc@v1.56.1/server.go:957 +0x18c

Error: The terraform-provider-xray_v2.0.0 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

And x-ray config of repository is like this:

terraform {
  required_providers {
    artifactory = {
      source  = "registry.terraform.io/jfrog/artifactory"
      version = ">= 6.10.1, < 9.0.0"
    }
    xray = {
      source  = "registry.terraform.io/jfrog/xray"
      version = ">= 1.11.1, <= 2.0.0"
    }
  }
}

resource "artifactory_local_maven_repository" "repo" {
  key                  = var.key
  repo_layout_ref      = var.layout_ref
  description          = "**MANAGED BY TERRAFORM** ${var.description}"
  notes                = "Owner: ${var.owner}"
  includes_pattern     = var.includes_pattern
  excludes_pattern     = var.excludes_pattern
  priority_resolution  = var.priority_resolution
  handle_releases      = var.handle_releases
  handle_snapshots     = var.handle_snapshots
  property_sets        = ["artifactory"]
  download_direct      = var.download_direct
  xray_index           = var.xray_index
  blacked_out          = var.blacked_out
  checksum_policy_type = var.checksum_policy_type
}

resource "xray_repository_config" "local-maven-xray-config" {
  provider  = xray
  repo_name = var.key
  config {
    retention_in_days   = 90
  }
twoodhouse commented 11 months ago

I'm seeing exactly the same thing as others on this. Upgrading an xray_repository_config from the provider version 1.14.2 to 1.15.0 or greater breaks this provider internally.

I'm on Artifactory 7.59.12, Xray 3.61.5

I just tried deleting the resources and recreating them with the latest version of the provider. Looks like that re-creation run is failing with "{"error":"Request payload is invalid as vuln contextual analysis config was not expected"}"

Possibly my version of Artifactory or Xray is too old?

alexhung commented 11 months ago

@twoodhouse

"{"error":"Request payload is invalid as vuln contextual analysis config was not expected"}"

That error message suggests you don't have JFrog Advanced Security enabled. I tested this against an internal instance that doesn't have JAS enabled and did not get the error. I'll take another look.

alexhung commented 11 months ago

@aserzhankou The error message in your message suggest a separate issue. Please open a separate GitHub issue.

twoodhouse commented 11 months ago

@alexhung, that makes sense; I'm only on the regular Enterprise license, so I don't have JAS.

alexhung commented 11 months ago

@twoodhouse ok, I managed to reproduce this issue. Working on a fix.

twoodhouse commented 11 months ago

Awesome! That's a fast turnaround. I'll try this out today.

alexhung commented 11 months ago

@twoodhouse It was a typical lifecycle of a bug. Take a long time to figure out what's wrong, and barely any time to fix the actual issue! 😄

malamin commented 11 months ago

I am happy to confirm that the upgrade to v.2.0.2 works fine :) thanks for resolving that!

On Wed, 1 Nov 2023, 16:23 Alex Hung, @.***> wrote:

@twoodhouse https://github.com/twoodhouse It was a typical lifecycle of a bug. Take a long time to figure out what's wrong, and barely any time to fix the actual issue! 😄

— Reply to this email directly, view it on GitHub https://github.com/jfrog/terraform-provider-xray/issues/141#issuecomment-1789150585, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADWONM3XHSOCM27H6QFPFHDYCJSPRAVCNFSM6AAAAAA53W5IK6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBZGE2TANJYGU . You are receiving this because you were mentioned.Message ID: @.***>