search

jfrog / terraform-provider-artifactory

Terraform provider to manage JFrog Artifactory
https://jfrog.com/artifactory
Apache License 2.0
272 stars 105 forks source link

Apply fails despite successful plan #33

Closed KrystianMarek closed 3 years ago

KrystianMarek commented 3 years ago

After updating an existing permission, single line update, I end up with this:

15:14:10  + terraform init
15:14:11  Initializing modules...
15:14:11  - core_local in modules/core/local
15:14:11  - core_local.local_repo_generator in modules/core/local_repo_generator
15:14:11  - core_remote in modules/core/remote
15:14:11  - core_remote.dockerhub-name in modules/core/remote_naming_convention
15:14:11  - core_remote.mirrors in modules/core/remote_repo_generator
15:14:11
15:14:11  Initializing the backend...
15:14:11
15:14:11  Initializing provider plugins...
15:14:11  - Finding jfrog/artifactory versions matching "~> 2.2.5"...
15:14:12  - Installing jfrog/artifactory v2.2.5...
15:14:13  - Installed jfrog/artifactory v2.2.5 (signed by a HashiCorp partner, key ID 6B219DCCD7639232)
15:14:13
15:14:13  Partner and community providers are signed by their developers.
15:14:13  If you'd like to know more about provider signing, you can read about it here:
15:14:13  https://www.terraform.io/docs/plugins/signing.html
15:14:13
15:14:13  Terraform has created a lock file .terraform.lock.hcl to record the provider
15:14:13  selections it made above. Include this file in your version control repository
15:14:13  so that Terraform can guarantee to make the same selections by default when
15:14:13  you run "terraform init" in the future.
15:14:13
15:14:13  Terraform has been successfully initialized!
15:14:13
15:14:13  You may now begin working with Terraform. Try running "terraform plan" to see
15:14:13  any changes that are required for your infrastructure. All Terraform commands
15:14:13  should now work.
15:14:13
15:14:13  If you ever set or change modules or backend configuration for Terraform,
15:14:13  rerun this command to reinitialize your working directory. If you forget, other
15:14:13  commands will detect it and remind you to do so if necessary.
## -- redacted -- ##
15:14:14
[Pipeline] sh
15:14:15  + terraform plan
15:14:23  module.core_local.artifactory_group.default-rw-local: Refreshing state... [id=default-rw-local]
15:14:23  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[4]: Refreshing state... [id=jenkins-ci-remote]
15:14:23  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[0]: Refreshing state... [id=helm-remote]
15:14:23  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[8]: Refreshing state... [id=npm-remote]
15:14:23  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[1]: Refreshing state... [id=gocenter-remote]
15:14:23  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[6]: Refreshing state... [id=jfrog-dependencies-remote]
15:14:23  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[10]: Refreshing state... [id=ghcr-remote]
15:14:23  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[9]: Refreshing state... [id=pypi-remote]
15:14:23  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[7]: Refreshing state... [id=netbeans-remote]
15:14:24  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[3]: Refreshing state... [id=jcenter-remote]
15:14:25  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[2]: Refreshing state... [id=imagej-remote]
15:14:25  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[5]: Refreshing state... [id=jenkins-ci-incrementals-remote]
15:14:25  module.core_remote.artifactory_group.default-cache-remote: Refreshing state... [id=default-cache-remote]
15:14:25  module.core_remote.artifactory_remote_repository.dockerhub: Refreshing state... [id=dockerhub-remote]
15:14:25  module.core_remote.artifactory_permission_target.default-permission-cache-remote: Refreshing state... [id=default-permission-cache-remote]
15:14:28  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[3]: Refreshing state... [id=default-maven-dev-local]
15:14:28  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[2]: Refreshing state... [id=default-docker-dev-local]
15:14:28  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[5]: Refreshing state... [id=default-pypi-dev-local]
15:14:28  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[7]: Refreshing state... [id=default-go-dev-local]
15:14:28  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[6]: Refreshing state... [id=default-conan-dev-local]
15:14:28  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[4]: Refreshing state... [id=default-helm-dev-local]
15:14:28  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[0]: Refreshing state... [id=default-npm-dev-local]
15:14:28  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[1]: Refreshing state... [id=default-generic-dev-local]
15:14:29  module.core_local.artifactory_permission_target.default-permission-rw-local: Refreshing state... [id=default-permission-rw-local]
15:14:29
15:14:29  An execution plan has been generated and is shown below.
15:14:29  Resource actions are indicated with the following symbols:
15:14:29    ~ update in-place
15:14:29
15:14:29  Terraform will perform the following actions:
15:14:29
15:14:29    # module.core_remote.artifactory_permission_target.default-permission-cache-remote will be updated in-place
15:14:29    ~ resource "artifactory_permission_target" "default-permission-cache-remote" {
15:14:29          id   = "default-permission-cache-remote"
15:14:29          name = "default-permission-cache-remote"
15:14:29
15:14:29        ~ repo {
15:14:29              # (3 unchanged attributes hidden)
15:14:29
15:14:29            ~ actions {
15:14:29                + groups {
15:14:29                    + name        = "default-cache-remote"
15:14:29                    + permissions = [
15:14:29                        + "annotate",
15:14:29                        + "delete",
15:14:29                        + "read",
15:14:29                        + "write",
15:14:29                      ]
15:14:29                  }
15:14:29                - groups {
15:14:29                    - name        = "default-cache-remote" -> null
15:14:29                    - permissions = [
15:14:29                        - "annotate",
15:14:29                        - "read",
15:14:29                        - "write",
15:14:29                      ] -> null
15:14:29                  }
15:14:29              }
15:14:29          }
15:14:29      }
15:14:29
15:14:29    # module.core_remote.artifactory_remote_repository.dockerhub will be updated in-place
15:14:29    ~ resource "artifactory_remote_repository" "dockerhub" {
15:14:29          id                                    = "dockerhub-remote"
15:14:29        ~ password                              = (sensitive value)
15:14:29          # (45 unchanged attributes hidden)
15:14:29
15:14:29        - content_synchronisation {
15:14:29            - enabled = false -> null
15:14:29          }
15:14:29      }
15:14:29
15:14:29    # module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[0] will be updated in-place
15:14:29    ~ resource "artifactory_remote_repository" "local-repo-from-template" {
15:14:29          id                                    = "helm-remote"
15:14:29          # (31 unchanged attributes hidden)
15:14:29
15:14:29        - content_synchronisation {
15:14:29            - enabled = false -> null
15:14:29          }
15:14:29      }
15:14:29
15:14:29    # module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[1] will be updated in-place
15:14:29    ~ resource "artifactory_remote_repository" "local-repo-from-template" {
15:14:29          id                                    = "gocenter-remote"
15:14:29          # (33 unchanged attributes hidden)
15:14:29
15:14:29        - content_synchronisation {
15:14:29            - enabled = false -> null
15:14:29          }
15:14:29      }
15:14:29
15:14:29    # module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[2] will be updated in-place
15:14:29    ~ resource "artifactory_remote_repository" "local-repo-from-template" {
15:14:29          id                                    = "imagej-remote"
15:14:29          # (31 unchanged attributes hidden)
15:14:29
15:14:29        - content_synchronisation {
15:14:29            - enabled = false -> null
15:14:29          }
15:14:29      }
15:14:29
15:14:29    # module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[3] will be updated in-place
15:14:29    ~ resource "artifactory_remote_repository" "local-repo-from-template" {
15:14:29          id                                    = "jcenter-remote"
15:14:29          # (31 unchanged attributes hidden)
15:14:29
15:14:29        - content_synchronisation {
15:14:29            - enabled = false -> null
15:14:29          }
15:14:29      }
15:14:29
15:14:29    # module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[4] will be updated in-place
15:14:29    ~ resource "artifactory_remote_repository" "local-repo-from-template" {
15:14:29          id                                    = "jenkins-ci-remote"
15:14:29          # (31 unchanged attributes hidden)
15:14:29
15:14:29        - content_synchronisation {
15:14:29            - enabled = false -> null
15:14:29          }
15:14:29      }
15:14:29
15:14:29    # module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[5] will be updated in-place
15:14:29    ~ resource "artifactory_remote_repository" "local-repo-from-template" {
15:14:29          id                                    = "jenkins-ci-incrementals-remote"
15:14:29          # (31 unchanged attributes hidden)
15:14:29
15:14:29        - content_synchronisation {
15:14:29            - enabled = false -> null
15:14:29          }
15:14:29      }
15:14:29
15:14:29    # module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[6] will be updated in-place
15:14:29    ~ resource "artifactory_remote_repository" "local-repo-from-template" {
15:14:29          id                                    = "jfrog-dependencies-remote"
15:14:29          # (31 unchanged attributes hidden)
15:14:29
15:14:29        - content_synchronisation {
15:14:29            - enabled = false -> null
15:14:29          }
15:14:29      }
15:14:29
15:14:29    # module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[7] will be updated in-place
15:14:29    ~ resource "artifactory_remote_repository" "local-repo-from-template" {
15:14:29          id                                    = "netbeans-remote"
15:14:29          # (31 unchanged attributes hidden)
15:14:29
15:14:29        - content_synchronisation {
15:14:29            - enabled = false -> null
15:14:29          }
15:14:29      }
15:14:29
15:14:29    # module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[8] will be updated in-place
15:14:29    ~ resource "artifactory_remote_repository" "local-repo-from-template" {
15:14:29          id                                    = "npm-remote"
15:14:29          # (31 unchanged attributes hidden)
15:14:29
15:14:29        - content_synchronisation {
15:14:29            - enabled = false -> null
15:14:29          }
15:14:29      }
15:14:29
15:14:29    # module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[9] will be updated in-place
15:14:29    ~ resource "artifactory_remote_repository" "local-repo-from-template" {
15:14:29          id                                    = "pypi-remote"
15:14:29          # (32 unchanged attributes hidden)
15:14:29
15:14:29        - content_synchronisation {
15:14:29            - enabled = false -> null
15:14:29          }
15:14:29      }
15:14:29
15:14:29    # module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[10] will be updated in-place
15:14:29    ~ resource "artifactory_remote_repository" "local-repo-from-template" {
15:14:29          id                                    = "ghcr-remote"
15:14:29          # (31 unchanged attributes hidden)
15:14:29
15:14:29        - content_synchronisation {
15:14:29            - enabled = false -> null
15:14:29          }
15:14:29      }
15:14:29
15:14:29  Plan: 0 to add, 13 to change, 0 to destroy.
15:14:29
15:14:29  ------------------------------------------------------------------------
15:14:29
15:14:29  Note: You didn't specify an "-out" parameter to save this plan, so Terraform
15:14:29  can't guarantee that exactly these actions will be performed if
15:14:29  "terraform apply" is subsequently run.
15:14:29
## -- redacted -- ##
15:14:30
[Pipeline] sh
15:14:30  + terraform apply -auto-approve
15:14:38  module.core_remote.artifactory_remote_repository.dockerhub: Refreshing state... [id=dockerhub-remote]
15:14:38  module.core_remote.artifactory_group.default-cache-remote: Refreshing state... [id=default-cache-remote]
15:14:38  module.core_remote.artifactory_permission_target.default-permission-cache-remote: Refreshing state... [id=default-permission-cache-remote]
15:14:39  module.core_local.artifactory_group.default-rw-local: Refreshing state... [id=default-rw-local]
15:14:39  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[0]: Refreshing state... [id=helm-remote]
15:14:39  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[3]: Refreshing state... [id=jcenter-remote]
15:14:39  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[6]: Refreshing state... [id=jfrog-dependencies-remote]
15:14:39  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[4]: Refreshing state... [id=jenkins-ci-remote]
15:14:39  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[1]: Refreshing state... [id=gocenter-remote]
15:14:39  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[5]: Refreshing state... [id=jenkins-ci-incrementals-remote]
15:14:39  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[8]: Refreshing state... [id=npm-remote]
15:14:39  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[10]: Refreshing state... [id=ghcr-remote]
15:14:39  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[2]: Refreshing state... [id=imagej-remote]
15:14:41  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[7]: Refreshing state... [id=netbeans-remote]
15:14:41  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[9]: Refreshing state... [id=pypi-remote]
15:14:45  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[1]: Refreshing state... [id=default-generic-dev-local]
15:14:45  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[3]: Refreshing state... [id=default-maven-dev-local]
15:14:45  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[2]: Refreshing state... [id=default-docker-dev-local]
15:14:45  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[7]: Refreshing state... [id=default-go-dev-local]
15:14:45  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[6]: Refreshing state... [id=default-conan-dev-local]
15:14:45  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[0]: Refreshing state... [id=default-npm-dev-local]
15:14:45  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[5]: Refreshing state... [id=default-pypi-dev-local]
15:14:45  module.core_local.module.local_repo_generator.artifactory_local_repository.local-repo-from-template[4]: Refreshing state... [id=default-helm-dev-local]
15:14:46  module.core_local.artifactory_permission_target.default-permission-rw-local: Refreshing state... [id=default-permission-rw-local]
15:14:54  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[9]: Modifying... [id=pypi-remote]
15:14:55  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[10]: Modifying... [id=ghcr-remote]
15:14:55  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[2]: Modifying... [id=imagej-remote]
15:14:55  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[0]: Modifying... [id=helm-remote]
15:14:55  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[8]: Modifying... [id=npm-remote]
15:14:55  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[4]: Modifying... [id=jenkins-ci-remote]
15:14:55  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[5]: Modifying... [id=jenkins-ci-incrementals-remote]
15:14:55  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[3]: Modifying... [id=jcenter-remote]
15:14:55  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[7]: Modifying... [id=netbeans-remote]
15:14:55  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[6]: Modifying... [id=jfrog-dependencies-remote]
15:14:56  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[9]: Modifications complete after 1s [id=pypi-remote]
15:14:56  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[1]: Modifying... [id=gocenter-remote]
15:14:56  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[8]: Modifications complete after 2s [id=npm-remote]
15:14:56  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[10]: Modifications complete after 2s [id=ghcr-remote]
15:14:56  module.core_remote.artifactory_permission_target.default-permission-cache-remote: Modifying... [id=default-permission-cache-remote]
15:14:57  module.core_remote.artifactory_permission_target.default-permission-cache-remote: Modifications complete after 0s [id=default-permission-cache-remote]
15:14:57  module.core_remote.artifactory_remote_repository.dockerhub: Modifying... [id=dockerhub-remote]
15:14:58  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[0]: Modifications complete after 3s [id=helm-remote]
15:14:58  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[6]: Modifications complete after 3s [id=jfrog-dependencies-remote]
15:14:58  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[2]: Modifications complete after 3s [id=imagej-remote]
15:14:59  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[5]: Modifications complete after 4s [id=jenkins-ci-incrementals-remote]
15:14:59  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[7]: Modifications complete after 4s [id=netbeans-remote]
15:14:59  module.core_remote.module.mirrors.artifactory_remote_repository.local-repo-from-template[1]: Modifications complete after 3s [id=gocenter-remote]
15:14:59  module.core_remote.artifactory_remote_repository.dockerhub: Modifications complete after 3s [id=dockerhub-remote]
15:15:05
15:15:05  Error: POST https://X_redacted_X/artifactory/api/repositories/jcenter-remote: 400 [{Status:400 Message:Could not merge and save new descriptor [org.jfrog.common.ExecutionFailed: Last retry failed: Should update revision 3108. Not trying again (exceeded number of attempts (5))]
15:15:05  }]
15:15:05
15:15:05    on modules/core/remote_repo_generator/main.tf line 33, in resource "artifactory_remote_repository" "local-repo-from-template":
15:15:05    33: resource "artifactory_remote_repository" "local-repo-from-template" {
15:15:05
15:15:05
15:15:05
15:15:05  Error: POST https://X_redacted_X/artifactory/api/repositories/jenkins-ci-remote: 400 [{Status:400 Message:Could not merge and save new descriptor [org.jfrog.common.ExecutionFailed: Last retry failed: Should update revision 3108. Not trying again (exceeded number of attempts (5))]
15:15:05  }]
15:15:05
15:15:05    on modules/core/remote_repo_generator/main.tf line 33, in resource "artifactory_remote_repository" "local-repo-from-template":
15:15:05    33: resource "artifactory_remote_repository" "local-repo-from-template" {
15:15:05
15:15:05
[Pipeline] }
15:15:05
chb0github commented 3 years ago

Let me get with the RT team to find out what that actually means

chb0github commented 3 years ago

Can I have the terraform file you used for this? I really want to run it locally.

chb0github commented 3 years ago

I need to know the version of RT you're running and the TF file you're executing. It looks like you're working with an already configured instance of RT. Just because the TF provide returns an error from RT doesn't mean the TF provider is the problem.

If I can't get a copy-and-paste reproducible case, I will need to close this ticket.

neilbeukes commented 3 years ago

We are also seeing this quite regularly, our solution was actually to limit the parallel option on Terraform to like 2 or 3. It is also more common the bigger amount of changes there are in the apply. It seems like a problem with the API not being able to keep up with the requests or some Async issues.

chb0github commented 3 years ago

I would not consider it an "problem" with the API - There is no obligation for an API to support full ACID - frequently, for reasons of performance, async processing is desired and occurs. Sometimes it's a function of a data storage choice basically dictating this behavior.

A good solution would be for the go code in the provider to actually implement retry/backoff.

An enhancement request would be that the provider support high concurrency interactions with RT.