search

RavinderReddyF5 / terraform-provider-bigip-version0.12

Terraform resources that can configure F5 BIGIP products
Mozilla Public License 2.0
0 stars 0 forks source link

Terraform Plan Times Out #181

Open RavinderReddyF5 opened 4 years ago

RavinderReddyF5 commented 4 years ago

Issue by spirrello Saturday Aug 17, 2019 at 18:13 GMT Originally opened as https://github.com/terraform-providers/terraform-provider-bigip/issues/150

When we run a Terraform plan against our LTM we're getting timeout errors. Is there a way to increase the timeout period? We're never able to establish a successful run of this script.

LTM 13.1.0.8 Build 0.0.3 Point Release 8 Provider version = "0.12.4" Terraform v0.12.6

5:57:08 + terraform plan -var-file=f5.secrets.tfvars -var-file=terraform.tfvars.json 15:57:09 Refreshing Terraform state in-memory prior to plan... 15:57:09 The refreshed state will be used to calculate this plan, but will not be 15:57:09 persisted to local or remote state storage. 15:57:09  15:57:09
15:57:09 ------------------------------------------------------------------------ 15:58:17  15:58:17 Error: Get https://10.207.129.32/mgmt/tm/net/self: net/http: request canceled (Client.Timeout exceeded while awaiting headers) 15:58:17
15:58:17  on provider.tf line 5, in provider "bigip": 15:58:17 5: provider "bigip" { 15:58:17  15:58:17 

RavinderReddyF5 commented 4 years ago

Comment by spirrello Monday Aug 19, 2019 at 14:06 GMT

By the way, I opened a case with F5 support and they asked the same question on if we can increase the timeout.

RavinderReddyF5 commented 4 years ago

Comment by dannyk81 Monday Aug 19, 2019 at 14:24 GMT

@spirrello really difficult to understand the output you've pasted, but this seems like you might have some connectivity issue between the machine running terraform and bigip.

are you able to run some manual curl commands against this bigip device? is port tcp/443 reachable on the bigip?

The above rest query just GETs the Self IPs, it shouldn't take more than a second, while in your case seems to timeout after ~1 minute (?)

This is the same rest query I did on a bigip device located >6000km from me:

$ curl -k -w "@curl-format.txt" -u admin:********* https://mad-bigip01.tuenti.mgm/mgmt/tm/net/self 
{"kind":"tm:net:self:selfcollectionstate","selfLink":"https://localhost/mgmt/tm/net/self?ver=12.1.1","items":....

    time_namelookup:  0.156028
       time_connect:  0.279073
    time_appconnect:  0.553400
   time_pretransfer:  0.553510
      time_redirect:  0.000000
 time_starttransfer:  0.754777
                    ----------
         time_total:  0.754919

:point_up: took less than 1 second overall

RavinderReddyF5 commented 4 years ago

Comment by spirrello Monday Aug 19, 2019 at 14:50 GMT

We don't have any connectivity issues and I've verified this by running packet captures on the LTM's management interface. I've verified that the LTM is taking approximately 9 seconds to respond after the TLS handshake and the Terraform client resets the connection after finally getting a response from the LTM.

image

RavinderReddyF5 commented 4 years ago

Comment by dannyk81 Monday Aug 19, 2019 at 15:05 GMT

I've verified that the LTM is taking approximately 9 seconds to respond after the TLS handshake

9 seconds for a TLS handshake is an awful lot of time... this shouldn't be taking this long and I would try to understand the root cause of this.

In my example this takes 0.5 second (time_appconnect: 0.553400).

time_appconnect The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed.

In any case, the go-bigip library sets a default timeout of 60 seconds for all API calls https://github.com/f5devcentral/go-bigip/blob/cb399c531a76f0c01442fd50d78def40341fb8db/bigip.go#L17-L19

hard to read you capture screenshot, since there's no time reference (only deltas) and no direction (from/to) indications... but judging by the 59.8.. delta there in the middle, looks like the request is taking a lot longer than 9 seconds.

have you tried to run a curl for the same Rest query and measure how long it takes, a -w flag using a format file like this:

    time_namelookup:  %{time_namelookup}\n
       time_connect:  %{time_connect}\n
    time_appconnect:  %{time_appconnect}\n
   time_pretransfer:  %{time_pretransfer}\n
      time_redirect:  %{time_redirect}\n
 time_starttransfer:  %{time_starttransfer}\n
                    ----------\n
         time_total:  %{time_total}\n

can help diagnose where time is spent during the request.

RavinderReddyF5 commented 4 years ago

Comment by spirrello Tuesday Aug 20, 2019 at 17:07 GMT

I have a case open with F5 and I'll share with what they come back with. Stay tuned.