search

aliyun / terraform-provider-alicloud

Terraform AliCloud provider
https://www.terraform.io/docs/providers/alicloud/
Mozilla Public License 2.0
590 stars 553 forks source link

OSS Buckets in Beijing region not creating from outside of china #1358

Open Omermiz opened 5 years ago

Omermiz commented 5 years ago

Hi there,

This issue only happens with Beijing region and outside of china, when trying to create resources from inside of china (using alicloud ECS instance, and running terraform from there).

Cannot create or refresh state of alicloud oss buckets with terraform only in Beijing region. I found the following insights: 

2019/06/25 23:48:05 [ERROR] root.oss_buckets: eval: *terraform.EvalSequence, err: 1 error occurred:
alicloud_oss_bucket.logging-bucket: Error creating OSS bucket: &url.Error{Op:"Put", URL:"https://cc-file-repository-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com/", Err:(*net.OpError)(0xc000146370)}

The URL caught my attention. Then I managed to find in the office document: https://www.alibabacloud.com/help/doc-detail/31837.htm?spm=a2c63.p38356.b99.24.3bc24f0d1SlaGw#title-9yy-k13-j8w

When using SDKs, use http:// or https:// + endpoint as the initialization parameter. For example, we recommend that you use http://oss-cn-shanghai.aliyuncs.com or https://oss-cn-shanghai.aliyuncs.com as the initialization parameter of an endpoint in China East 2 (Shanghai). Do not use a third-level domain name, that is, http://bucket.oss-cn-shanghai.aliyuncs.com, as the initialization parameter.

So I guess this is the cause. Actually oss-cn-beijing.aliyuncs.com and <bucket>.oss-cn-beijing.aliyuncs.com resolve to different IP. I suspect that Aliyun handle them differently. So I did a experiment: edit the hosts file to let <bucket>.oss-cn-beijing.aliyuncs.com to resolve to the same IP as oss-cn-beijing.aliyuncs.com, which is 59.110.190.54 or 59.110.190.69:

59.110.190.54 cc-dump-files-tttt-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com
59.110.190.54 cc-platform-backends-manager-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com
59.110.190.54 cc-file-repository-logs-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com
59.110.190.54 cc-file-repository-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com

The terraform run successfully.

Terraform Version

terraform -v Terraform v0.11.14

Affected Resource(s)

Terraform Configuration Files

resource "alicloud_oss_bucket" "bucket"{
  bucket = "test-bucket-oss-terraform"
  acl = "private"
}

Debug Output

https://gist.github.com/Omermiz/ce97ae7a7594e79a80903ea762777a6c

Expected Behavior

Buckets should be created, or at least buckets state should be refreshed.

Actual Behavior

timeouts from oss service only in beijing region.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

  1. terraform apply
huiguangjun commented 5 years ago

Can you help to test it with the ossutil tool (download from https://www.alibabacloud.com/help/doc-detail/50452.htm) ? the cmd : 1) use https: ossutil64 -e https://oss-cn-beijing.aliyuncs.com -i access_key_id -k access_key_secret stat oss://cc-dump-files-cn-beijing-dv1 --loglevel=debug

2) use http:
ossutil64 -e http://oss-cn-beijing.aliyuncs.com -i access_key_id -k access_key_secret stat oss://cc-dump-files-cn-beijing-dv1 --loglevel=debug

Omermiz commented 5 years ago

where can i post the log file? my access key and secret are in the logs.

huiguangjun commented 5 years ago

you can remove your access key and secret first, then post the log here.

Omermiz commented 5 years ago

Here is the log:

2019/07/02 10:59:34 [info]ossutil run begin,cmd:ossutilmac64 -e https://oss-cn-beijing.aliyuncs.com -i access_key -k secret_access_key stat oss://cc-dump-files-cn-beijing-alta-sqa1 --loglevel=debug
2019/07/02 10:59:34 [info]oss go sdk version is v1.9.8
2019/07/02 10:59:34 [info]go version is go1.11.8
2019/07/02 10:59:34 [info]runtime.NumCPU 12
2019/07/02 10:59:34 [debug][Req:0xc000180100]signStr:GET

Tue, 02 Jul 2019 07:59:34 GMT
/cc-dump-files-cn-beijing-alta-sqa1/?bucketInfo.
2019/07/02 10:59:34 [debug][Req:0xc000180100]Method:GET Host:cc-dump-files-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com Path:/  Query:bucketInfo    Header info:    Host:https://oss-cn-beijing.aliyuncs.com    User-Agent:aliyun-sdk-go/v1.9.8 (Darwin/18.6.0/x86_64;go1.11.8)/ossutil-v1.6.1  Authorization:OSS access_key:6R+l0HsyP6rqb+NjUW2B156dNDo=   Content-Length:0    Date:Tue, 02 Jul 2019 07:59:34 GMT
2019/07/02 11:00:50 [debug][Req:0xc000228000]signStr:GET

Tue, 02 Jul 2019 08:00:50 GMT
/cc-dump-files-cn-beijing-alta-sqa1/?bucketInfo.
2019/07/02 11:00:50 [debug][Req:0xc000228000]Method:GET Host:cc-dump-files-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com Path:/  Query:bucketInfo    Header info:    Content-Length:0    Date:Tue, 02 Jul 2019 08:00:50 GMT  Host:https://oss-cn-beijing.aliyuncs.com    User-Agent:aliyun-sdk-go/v1.9.8 (Darwin/18.6.0/x86_64;go1.11.8)/ossutil-v1.6.1  Authorization:OSS access_key:qfW2mK5YSdX5T3YvGvAR5GLAL3o=
2019/07/02 11:02:06 [debug][Req:0xc000180200]signStr:GET

Tue, 02 Jul 2019 08:02:06 GMT
/cc-dump-files-cn-beijing-alta-sqa1/?bucketInfo.
2019/07/02 11:02:06 [debug][Req:0xc000180200]Method:GET Host:cc-dump-files-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com Path:/  Query:bucketInfo    Header info:    User-Agent:aliyun-sdk-go/v1.9.8 (Darwin/18.6.0/x86_64;go1.11.8)/ossutil-v1.6.1  Authorization:OSS access_key:rakGBxpxyYyUqnEObYnFdv/V6xw=   Content-Length:0    Date:Tue, 02 Jul 2019 08:02:06 GMT  Host:https://oss-cn-beijing.aliyuncs.com
2019/07/02 11:03:22 [debug][Req:0xc000284100]signStr:GET

Tue, 02 Jul 2019 08:03:22 GMT
/cc-dump-files-cn-beijing-alta-sqa1/?bucketInfo.
2019/07/02 11:03:22 [debug][Req:0xc000284100]Method:GET Host:cc-dump-files-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com Path:/  Query:bucketInfo    Header info:    Authorization:OSS access_key:TkDymSzLOSkzSnT0bGuxayOgCaU=   Content-Length:0    Date:Tue, 02 Jul 2019 08:03:22 GMT  Host:https://oss-cn-beijing.aliyuncs.com    User-Agent:aliyun-sdk-go/v1.9.8 (Darwin/18.6.0/x86_64;go1.11.8)/ossutil-v1.6.1
2019/07/02 11:03:43 [debug][Resp:0xc000284100]StatusCode:200    Header info:    Content-Type:application/xml    Content-Length:789  Connection:keep-alive   X-Oss-Request-Id:5D1B0FDF3243A9928DC60697   X-Oss-Server-Time:45    Server:AliyunOSS    Date:Tue, 02 Jul 2019 08:03:43 GMT
2019/07/02 11:03:43 [info]ossutil run end,cost :248757(ms).
2019/07/02 11:12:00 [info]ossutil run begin,cmd:ossutilmac64 -e http://oss-cn-beijing.aliyuncs.com -i access_key -k secret_access_key stat oss://cc-dump-files-cn-beijing-alta-sqa1 --loglevel=debug
2019/07/02 11:12:00 [info]oss go sdk version is v1.9.8
2019/07/02 11:12:00 [info]go version is go1.11.8
2019/07/02 11:12:00 [info]runtime.NumCPU 12
2019/07/02 11:12:00 [debug][Req:0xc000182100]signStr:GET

Tue, 02 Jul 2019 08:12:00 GMT
/cc-dump-files-cn-beijing-alta-sqa1/?bucketInfo.
2019/07/02 11:12:00 [debug][Req:0xc000182100]Method:GET Host:cc-dump-files-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com Path:/  Query:bucketInfo    Header info:    Content-Length:0    Date:Tue, 02 Jul 2019 08:12:00 GMT  Host:http://oss-cn-beijing.aliyuncs.com User-Agent:aliyun-sdk-go/v1.9.8 (Darwin/18.6.0/x86_64;go1.11.8)/ossutil-v1.6.1  Authorization:OSS access_key:aa3gdwRsk6njGDXsmzMvfgUDPDY=
2019/07/02 11:17:26 [debug][Req:0xc00024a000]signStr:GET

Tue, 02 Jul 2019 08:17:26 GMT
/cc-dump-files-cn-beijing-alta-sqa1/?bucketInfo.
2019/07/02 11:17:26 [debug][Req:0xc00024a000]Method:GET Host:cc-dump-files-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com Path:/  Query:bucketInfo    Header info:    Content-Length:0    Date:Tue, 02 Jul 2019 08:17:26 GMT  Host:http://oss-cn-beijing.aliyuncs.com User-Agent:aliyun-sdk-go/v1.9.8 (Darwin/18.6.0/x86_64;go1.11.8)/ossutil-v1.6.1  Authorization:OSS access_key:GOW5UM6O42M1GS262+Jzk7vtNZQ=
2019/07/02 11:18:42 [debug][Req:0xc00024a100]signStr:GET

Tue, 02 Jul 2019 08:18:42 GMT
/cc-dump-files-cn-beijing-alta-sqa1/?bucketInfo.
2019/07/02 11:18:42 [debug][Req:0xc00024a100]Method:GET Host:cc-dump-files-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com Path:/  Query:bucketInfo    Header info:    Date:Tue, 02 Jul 2019 08:18:42 GMT  Host:http://oss-cn-beijing.aliyuncs.com User-Agent:aliyun-sdk-go/v1.9.8 (Darwin/18.6.0/x86_64;go1.11.8)/ossutil-v1.6.1  Authorization:OSS access_key:q7Zidc7GH37yTQzYBPc0VBipNuc=   Content-Length:0
2019/07/02 11:24:04 [debug][Req:0xc000276000]signStr:GET

Tue, 02 Jul 2019 08:24:04 GMT
/cc-dump-files-cn-beijing-alta-sqa1/?bucketInfo.
2019/07/02 11:24:04 [debug][Req:0xc000276000]Method:GET Host:cc-dump-files-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com Path:/  Query:bucketInfo    Header info:    Date:Tue, 02 Jul 2019 08:24:04 GMT  Host:http://oss-cn-beijing.aliyuncs.com User-Agent:aliyun-sdk-go/v1.9.8 (Darwin/18.6.0/x86_64;go1.11.8)/ossutil-v1.6.1  Authorization:OSS access_key:i5aEg+MiLUw8G3gidQkdsqK7GsA=   Content-Length:0
2019/07/02 11:25:20 [debug][Req:0xc00024a200]signStr:GET

Tue, 02 Jul 2019 08:25:20 GMT
/cc-dump-files-cn-beijing-alta-sqa1/?bucketInfo.
2019/07/02 11:25:20 [debug][Req:0xc00024a200]Method:GET Host:cc-dump-files-cn-beijing-alta-sqa1.oss-cn-beijing.aliyuncs.com Path:/  Query:bucketInfo    Header info:    Content-Length:0    Date:Tue, 02 Jul 2019 08:25:20 GMT  Host:http://oss-cn-beijing.aliyuncs.com User-Agent:aliyun-sdk-go/v1.9.8 (Darwin/18.6.0/x86_64;go1.11.8)/ossutil-v1.6.1  Authorization:OSS access_key:kw62sPRMYbtPhos7trV5Sn63uwI=
2019/07/02 11:25:41 [debug][Resp:0xc00024a200]StatusCode:200    Header info:    Date:Tue, 02 Jul 2019 08:25:41 GMT  Content-Type:application/xml    Content-Length:789  Connection:keep-alive   X-Oss-Request-Id:5D1B15053772E52EDCF8AC4A   X-Oss-Server-Time:64    Server:AliyunOSS
2019/07/02 11:25:41 [info]ossutil run end,cost :820916(ms).
Omermiz commented 5 years ago

Anything new with this?