search

FlexibleEngineCloud / terraform-provider-flexibleengine

Terraform flexibleengine provider
https://www.terraform.io/docs/providers/flexibleengine/
Mozilla Public License 2.0
30 stars 53 forks source link

[ELB] internal elb timeout although it is created #404

Closed amyounis closed 3 years ago

amyounis commented 4 years ago

Terraform Version

0.13.5

Terraform Configuration Files

provider "flexibleengine" {
  user_name   = var.user_name
  password    = var.password
  domain_name = var.domain_name
  tenant_name = var.tenant_name
  auth_url    = "https://iam.eu-west-0.prod-cloud-ocb.orange-business.com/v3"
  region      = var.region
  version     = "~> 1.16"
}

.
.
.

resource "flexibleengine_elb_loadbalancer" "elb_2" {
  count = var.enable_autoscaling ? 1 : 0
  name              = "rancher-elb"
  type              = "Internal"
  description       = "rancher elb"
  vpc_id            = flexibleengine_vpc_v1.vpc_1.id
  security_group_id = flexibleengine_networking_secgroup_v2.secgroup_2.id
  vip_subnet_id     = flexibleengine_vpc_subnet_v1.subnet_1.id
  az                = "eu-west-0a"
  admin_state_up    = true
}
.
.
.

Debug Output

https://gist.github.com/amyounis/aa9107bef72945cff75d127ef6d31173

Expected Behavior

create elb resource and continue

Actual Behavior

the elb creation request is sent then terraform will query the status of the creation job for about 4 minutes then no queries sent after that. and although the elb is actually created, the terraform will not know and after sometime (depending on timeouts settings) it will give the timeout error

Additional Context

other resources (like rds) take long time to be created but it is working OK and it doesn't behave the same as elb. also, elb resource with type "External" (meaning it has a public IP) like this: resource "flexibleengine_elb_loadbalancer" "elb_1" { name = "k8s-workers-elb" type = "External" bandwidth = flexibleengine_vpc_eip_v1.eip_nat[1].bandwidth.0.size description = "kubernetes workers elb" vpc_id = flexibleengine_vpc_v1.vpc_1.id vip_address = flexibleengine_vpc_eip_v1.eip_nat[1].publicip.0.ip_address admin_state_up = true } is working OK and it is created without a problem !!!

amyounis commented 4 years ago

no one faced this !

amyounis commented 4 years ago

the same issue occurred once with DCS resource when it took too long to be created.

amyounis commented 3 years ago

this is frustrating :(

amyounis commented 3 years ago

very frustrating

antonin-a commented 3 years ago

Hello @amyounis sorry to see that frustration, I assume that you are still facing the issue... @niuzhenguo and @DafuSHI can you please check this issue ?

niuzhenguo commented 3 years ago

@amyounis Sorry for the late response, we have a default timeout for create/update/delete in these resources. You can customize the timeout by adding a timeout block in the resource defination as bellow. And we can bump the timeout value if it takes more time then the default value.

resource "flexibleengine_elb_loadbalancer" "elb_1" { name = "k8s-workers-elb" .... timeouts { create = "5m" } }

amyounis commented 3 years ago

@niuzhenguo i already tried "timeout" option , and the behavior is the same except it will take more time before timeout. I mentioned that in the Actual Behavior section: " Actual Behavior the elb creation request is sent then terraform will query the status of the creation job for about 4 minutes then no queries sent after that. and although the elb is actually created, the terraform will not know and after sometime (depending on timeouts settings) it will give the timeout error "

niuzhenguo commented 3 years ago

@amyounis I'm sorry I don't understand why it stopped querying the status of the creation job after 4m, is there any issue with the query request API? Can you please paste some logs about this as the debug output you provided on gist seems be truncated.

amyounis commented 3 years ago

@niuzhenguo @DafuSHI
query API is OK, I ran myself the API requests using curl while the terraform was running and it worked just fine, I also contacted flexible engine support and they confirmed that API is OK their side. you can show the whole trace by clicking "view the full file", it is 145235 lines. to see all interactions with ELB API just filter with "https://elb" you will find in the trace the following: 1- there are two internal ELB that is requested to be created, one with the name "k82-master-elb" and the other with the name "rancher-elb". 2- two POST requests were sent for the two ELBs, the first at line number 34852 and the other at line 35008. 3- the job IDs for creating the two ELBs are "ff808082742166a401758827bb071274" and "ff8080827421681101758827bb0c1b2d". 4- querying for the first job status started at 08:51:46 and ended at 08:54:21 (3 minutes and 25 seconds) with total number of 75 requests sent. 5- querying for the second job status started at 08:51:46 and ended at 08:54:22 (3 minutes and 24 seconds) with total number of 75 requests sent. 6- both responses to the last requests indicate that the job status is still "RUNNING" . 7- no queries sent after that to check the job status. 8- timeout error appear for both ELBs in 09:04:21 (line 142326) and 09:04:22 (line 142349) , about 6 minutes after last query request.

amyounis commented 3 years ago

@niuzhenguo @DafuSHI query API is OK, I ran myself the API requests using curl while the terraform was running and it worked just fine, I also contacted flexible engine support and they confirmed that API is OK their side. you can show the whole trace by clicking "view the full file", it is 145235 lines. to see all interactions with ELB API just filter with "https://elb" you will find in the trace the following: 1- there are two internal ELB that is requested to be created, one with the name "k82-master-elb" and the other with the name "rancher-elb". 2- two POST requests were sent for the two ELBs, the first at line number 34852 and the other at line 35008. 3- the job IDs for creating the two ELBs are "ff808082742166a401758827bb071274" and "ff8080827421681101758827bb0c1b2d". 4- querying for the first job status started at 08:51:46 and ended at 08:54:21 (3 minutes and 25 seconds) with total number of 75 requests sent. 5- querying for the second job status started at 08:51:46 and ended at 08:54:22 (3 minutes and 24 seconds) with total number of 75 requests sent. 6- both responses to the last requests indicate that the job status is still "RUNNING" . 7- no queries sent after that to check the job status. 8- timeout error appear for both ELBs in 09:04:21 (line 142326) and 09:04:22 (line 142349) , about 6 minutes after last query request.

niuzhenguo commented 3 years ago

@amyounis I still didn't see what's the problem, but I can update the WaitFor codes to make it issues less query APIs, please try laster.

niuzhenguo commented 3 years ago

Another thing I want to mention here is that, the Classic ELB seems already deprecated now, maybe you can try Enhanced ELB(flexibleengine_lb_xxx)

amyounis commented 3 years ago

@amyounis I still didn't see what's the problem, but I can update the WaitFor codes to make it issues less query APIs, please try laster. @niuzhenguo is there a limit on how many query requests sent to check job status ? is it configurable ? the time between sending query requests, is it configurable ?

niuzhenguo commented 3 years ago

@amyounis It's not configurable, please see the logic here https://github.com/huaweicloud/golangsdk/blob/master/util.go#L17 we already added a 5s sleep, but doesn't take effect on the provider now.

niuzhenguo commented 3 years ago

@DafuSHI @antonin-a Regarding the cost of classic LB and enhanced LB, can you explain here?

antonin-a commented 3 years ago

Hello @amyounis and @niuzhenguo , Yes, prices are exactly the same and can be found here : https://cloud.orange-business.com/nos-tarifs/elastic-load-balancer/ I confirm that we recommend to use Enhance ELB as Classic one tend to be deprecated.

qukuijin1989 commented 3 years ago

hi @amyounis , could you try Enhance ELB?

ShiChangkuo commented 3 years ago

@amyounis I'm going to close this issue because there is no updates for a long time.

If you have found a problem that seems similar to this, please feel free to open a new issue.