Job modify index changing on server side causing drift

[automagic]

Describe what happened

Pulumi is tracking the job output property modify_index in state, but this field changes often on the server side in Kubernetes for a variety of reasons, and is not really a field that should be considered "desired" in state. It is likely that this field should be ignored in most cases.

Sample program

const jobResource = new nomad.Job("jobResource", {
    jobspec: "string",
    detach: false,
    deregisterOnIdChange: false,
    consulToken: "string",
    hcl1: false,
    hcl2: {
        allowFs: false,
        vars: {
            string: "any",
        },
    },
    deregisterOnDestroy: false,
    json: false,
    policyOverride: false,
    purgeOnDestroy: false,
    rerunIfDead: false,
    vaultToken: "string",
});

Log output

https://github.com/hashicorp/terraform-provider-nomad/issues/398

Affected Resource(s)

https://www.pulumi.com/registry/packages/nomad/api-docs/job/

Output of pulumi about

latest

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[VenelinMartinov]

HI @automagic, thanks for reporting the issue and sorry you had trouble here. Did you try using ignoreChanges to ignore that field: https://www.pulumi.com/docs/iac/concepts/options/ignorechanges/

[sam-myers]

Hi @VenelinMartinov, I ran into this issue. Adding modifyIndex to ignoreChanges unfortunately did not help.

My usage (slightly simplified), Golang SDK:

job, err := nomad.NewJob(ctx, "nomad-job", &nomad.JobArgs{
    Jobspec: ...
}, pulumi.IgnoreChanges([]string{"modifyIndex"}))

Error when running pulumi up:

     Type                 Name                  Status         Info
     pulumi:pulumi:Stack  developer-portal-dev                 
     └─ nomad:index:Job   nomad-job-us-east-2   **failed**     1 error

Diagnostics:
  nomad:index:Job (nomad-job-us-east-2):
    error: diffing urn:pulumi:dev::developer-portal::nomad:index/job:Job::nomad-job-us-east-2: job modify index has changed since last refresh

Doing a pulumi refresh temporarily eases the issue by synchronizing the index, but it generates no diff:

SummaryDiffDiagnostic
     Type                             Name                  Plan     
     pulumi:pulumi:Stack              developer-portal-dev           
     ├─ nomad:index:Job               nomad-job-us-east-2                          
     ├─ pulumi:providers:nomad        nomad-provider                 
     └─ ...

Resources:
    14 unchanged

[automagic]

@VenelinMartinov note that error is coming from the TF provider, and discussed a bit here: https://github.com/hashicorp/terraform-provider-nomad/issues/398

[VenelinMartinov]

That's very unfortunate. I don't think we can do much on our side. I've updated the upstream issue to ask for a flag to disable the check.

The workaround for pulumi users facing this issue would be to run pulumi with --refresh=true.