Intermittent errors refreshing state for datafeed

jstangroome commented 5 years ago

Terraform Version

terraform:0.11.1

- Downloading plugin for provider "ns1" (1.1.0)...

Affected Resource(s)

ns1_datafeed

Terraform Configuration Files

resource "ns1_datafeed" "bar" {
  name      = "bar"
  source_id = "f83b8420d35d7a740d91458cba65865f"
  config {
    alarm_name = "the_alarm"
  }
}

Output

ns1_datafeed.bar: Refreshing state... (ID: 5c3d855cc9c79d0001873391)

Error: Error refreshing state: 1 error(s) occurred:

* module.foo.ns1_datafeed.bar: 1 error(s) occurred:

* module.foo.ns1_datafeed.bar: ns1_datafeed.bar: invalid character '<' looking for beginning of value

Expected Behavior

What should have happened? The reason (or at least the HTTP status code) why the NS1 API call failed should have been displayed instead of the error about <. For non-side-effect actions (like querying current state), failed API calls should be retried after a short pause before failing the entire terraform run.

Actual Behavior

What actually happened? The NS1 API call failed and returned a HTML response body error instead of the expected JSON response. The provider attempted to JSON-parse the response body but failed due to the <html> tag. The provider then aborted the terraform run.

Steps to Reproduce

Since the NS1 API was failing intermittently, reproduction will likely require intercepting the HTTP requests to NS1 to inject these error responses.

pcatalini commented 3 years ago

Hey @jstangroome thanks for reporting we added to our backlog

eravin-ns1 commented 1 year ago

Hello @jstangroome, can you advise whether you've seen this problem with recent versions of the provider?

bryanhiestand commented 1 year ago

I still see this issue right now with v1.12.7 and v1.13.0.

When the API returns a 502, the response is html instead of json. The terraform provider fails to parse the JSON, and we get this unhelpful error:

Error: GET https://api.nsone.net/v1/zones/ZONE/FQDN/CNAME: 502 invalid character '<' looking for beginning of value

When your API servers are having an issue (which I see right now), I can also reproduce 😄

curl  -X GET -H "X-NSONE-Key: ${APIKEY}" https://api.nsone.net/v1/zones/ZONE/FQDN/A

<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]>    <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>

<title>api.nsone.net | 502: Bad gateway</title>
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta name="robots" content="noindex, nofollow" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
<link rel="stylesheet" id="cf_styles-css" href="/cdn-cgi/styles/main.css" />

</head>
<body>
<div id="cf-wrapper">

    <div id="cf-error-details" class="p-0">
        <header class="mx-auto pt-10 lg:pt-6 lg:px-8 w-240 lg:w-full mb-8">
            <h1 class="inline-block sm:block sm:mb-2 font-light text-60 lg:text-4xl text-black-dark leading-tight mr-2">

              <span class="inline-block">Bad gateway</span>
              <span class="code-label">Error code 502</span>
            </h1>
            <div>
               Visit <a href="https://www.cloudflare.com/5xx-error-landing?utm_source=errorcode_502&utm_campaign=api.nsone.net" target="_blank" rel="noopener noreferrer">cloudflare.com</a> for more information.
            </div>
            <div class="mt-3">2022-12-07 01:12:50 UTC</div>
        </header>

        <div class="my-8 bg-gradient-gray">
            <div class="w-240 lg:w-full mx-auto">
                <div class="clearfix md:px-8">
...(snip)

eravin-ns1 commented 1 year ago

We've added retries for 502/503 errors in release 1.13.2-pre2. This should dramatically reduce the occurrence of this problem.

However, this doesn't address the underlying issue, where the HTTP error response can't be displayed because it doesn't parse as JSON. The workaround (starting in 1.13.2-pre1) is to run with TF_LOG=debug and NS1_DEBUG=yes set which will show the full HTTP response, but that's not very helpful for intermittent errors.

I'll keep this issue open to see if there's a way to get those errors properly passed upwards into Terraform.

eravin-ns1 commented 1 year ago

Turns out this had to get fixed in the ns1-go module. This has been included in release 1.13.2-pre4. HTTP errors should now show up in the Terraform output with the body of the message instead of the JSON parsing error.

ns1-terraform / terraform-provider-ns1