Error making requests to Cloudflare

[bootswithdefer]

Terraform CLI and Provider Versions

Terraform v1.5.6 on linux_amd64

• provider registry.terraform.io/hashicorp/http v3.4.1

Terraform Configuration

data "http" "example" {
  url = "https://www.cloudflare.com/ips-v4"
}

Expected Behavior

No errors

Actual Behavior

data.http.example: Reading...
╷
│ Error: Error making request
│ 
│   with data.http.example,
│   on main.tf line 1, in data "http" "example":
│    1: data "http" "example" {
│ 
│ Error making request: GET https://www.cloudflare.com/ips-v4 giving up after 1 attempt(s)

Steps to Reproduce

1. terraform apply

How much impact is this issue causing?

Medium

Logs

https://gist.github.com/bootswithdefer/0ac09cc56826691aec9427e33a7739e1

Additional Information

When querying the Cloudflare URL the provider errors, however if I disable the http2client (GODEBUG http2debug=1,http2client=0) then the provider is successful. This just started today as near as I can tell, we have been querying Cloudflare like this for years.

I tried a simple golang http client program and it can query Cloudflare with no issues. I tried building the provider locally with golang 1.21.6 and it also errors like the distributed provider.

The gist contains two outputs, one with GODEBUG http2debug=1,http2client=1 and the second with GODEBUG http2debug=1,http2client=0

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[acdha]

I just encountered this using the same URL for the same reason, with identical behaviour using Terraform Terraform v1.7.4 on darwin_arm64 and linux_amd64 both with provider registry.terraform.io/hashicorp/http v3.4.1. I also tested with a different Go-based HTTP client w/o issue and all other tools (curl, httpie, browsers, etc.) do not have this problem.

I deployed the workaround of setting GODEBUG to http2client=0 in our CI configuration which resolved this. I'm sure it's slightly slower but for two requests that's moot.

[samm-git]

It is enough to remove www. and it will start working.

[acdha]

It is enough to remove www. and it will start working.

That's really interesting because without www. it redirects to the original URL:

https://redbot.org/?uri=https://cloudflare.com/ips-v4

Location: https://www.cloudflare.com/ips-v4

[acdha]

Some regression testing shows that the last working version of the HTTP provider was 3.2.1 but 3.3.0 was a fairly large release with some significant changes like going from Go 1.18 to 1.19:

https://github.com/hashicorp/terraform-provider-http/compare/v3.2.1...v3.3.0

[bflad]

Hi everyone 👋 Thank you for the reports and sorry for the frustrating experience. The data source should be doing a better job returning more error information than just the failed HTTP method and URL.

As far as I can tell, the actual error occurring is a 500 Internal Server Error for these requests. Through a process of elimination, it appears related to the fact that the current data source implementation always sends an empty string request body, instead of only sending a request body when one is configured. While the receiving server should probably be ignoring that request body as it always was previously or sending a different 4xx error, the fix here (to be verified still) will likely be a small code change and a new provider release.

I will keep this issue updated with more details throughout the process. Thank you.

[bflad]

v3.4.2 has been released with a fix for this and should become available in the public Terraform Registry shortly.

[bootswithdefer]

3.4.2 resolved the issue for me.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.