newrelic_one_dashboard fails on large dashboards

[querry43]

Terraform Version

Terraform v1.2.5
on darwin_arm64
+ provider registry.terraform.io/fastly/fastly v3.0.4
+ provider registry.terraform.io/hashicorp/aws v4.51.0
+ provider registry.terraform.io/newrelic/newrelic v3.13.0

Affected Resource(s)

• newrelic_one_dashboard

Terraform Configuration

provider newrelic {
  region = "US"
}

resource newrelic_one_dashboard site_health_dashboard {
  name = "${title(module.globals.env)} Site Health"

  permissions = "public_read_write"

  page {
    name = "Site Status"

    widget_markdown {
      title  = ""
      row    = 1
      column = 1
      height = 2

      text = <<EOF
...
EOF
    }

    widget_markdown { ... }
    widget_line { ... }
  }
  page {
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
    widget_markdown { ... }
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
  }
  page {
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
  }
  page {
    widget_line { ... }
    widget_line { ... }
  }
  page {
    widget_line { ... }
    widget_line { ... }
    widget_line { ... }
  }
    page {
    widget_billboard { ... }
    widget_line { ... }
    widget_markdown { ... }
    widget_line { ... }
  }
  page {
    widget_billboard { ... }
    widget_pie { ... }
    widget_line { ... }
    widget_markdown { ... }
    widget_line { ... }
  }
  page {
    widget_pie { ... }
    widget_area { ... }
    widget_area { ... }
    widget_markdown { ... }
  }
  page {
    widget_markdown { ... }
    widget_table { ... }
    widget_markdown { ... }
    widget_line { ... }
    widget_line { ... }
  }

Actual Behavior

│ Error: maximum retries reached: An error occurred resolving this field
│ 
│   with newrelic_one_dashboard.site_health_dashboard,
│   on newrelic-site-health-dashboard.tf line 159, in resource "newrelic_one_dashboard" "site_health_dashboard":
│  159: resource newrelic_one_dashboard site_health_dashboard {

It is possible to create the dashboard by commenting out 2/3 of the pages and applying, then repeating for larger and larger segments of the dashboard.

Expected Behavior

The dashboard should be created or modified successfully

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply

[usfdmiller]

We, oddly enough, ran into this issue yesterday as well. We were trying to add a new page to an existing dashboard that is decently sized and received the exact same error when trying to update. However, if we delete and recreate the dashboard with the new page it works.

This is the response received from the API when it fails.

{
 "data": {
  "dashboardUpdate": null
 },
 "errors": [
  {
   "extensions": {
    "errorClass": "SERVER_ERROR"
   },
   "locations": [
    {
     "column": 5,
     "line": 4
    }
   ],
   "message": "An error occurred resolving this field",
   "path": [
    "dashboardUpdate"
   ]
  }
 ]
}

[usfdmiller]

I was able sanitize the dashboard with which I've been working to help duplicate this issue.

https://gist.github.com/usfdmiller/3cf82452b79713bd66439aa75fd1ae06

Now, if I apply this configuration "as is" there aren't be any errors. The dashboard and all pages are created. If I remove the page titled "Infra Performance", create the dashboard, then try to add that page back to the configuration after the "ClickStreams" page, I receive the above error during the update.

However, and this is a weird one, if I add the "Infra Performance" page back to the configuration before the "ClickStreams" page, there is no error during the update.

[kidk]

Thanks for details @usfdmiller I'll try to figure out what's going wrong.

[tcgbrett]

I also just ran into the same issue. Thanks for posting

[kidk]

I was able to reproduced it, and then it worked again 5 minutes later. I'm talking to the API team to see if we can figure this out.

Can you all do a double check if you're not hitting the limits documented here: https://docs.newrelic.com/docs/apis/nerdgraph/examples/nerdgraph-dashboards/#limits

[querry43]

I am not anywhere near those limits.

[kidk]

Small update: It's a weird bug, so more teams need to get involved. We're able to reproduce it consistently thanks to your message @tcgbrett Thanks a lot for putting in the time to help us reproduce, it's really appreciated.

[kidk]

New release v3.15.0 is out. 🥳

Can everyone check if this resolves the issue?

[kidk]

I'm going to close the issue as it should be resolved. Let me know if it's not the case.

[querry43]

I was able to confirm the issue with version 3.13 and the fix with version 3.15. Thanks!

[tcgbrett]

Just an FYI @querry43, I missed this update for weeks because I was waiting to see it in the release notes. I'm glad I accidentally stumbled back onto this issue. - It's not present here

[kidk]

I can understand why that's confusing, the fix was one_dashboard: fix for updating with filter_current_dashboard.