[Alibaba] signal SIGSEGV: segmentation violation when calling ListRegionZone()

[sykim-etri]

같은 서버에서 실행 중인 cb-tumblebug의 update-cloudinfo.sh 실행 중 아래와 같은 에러가 발생하면서 프로세스가 종료되는 증상이 나타납니다.

• commit: a97d68cf546b0dfcd70e1523a0387b4258485cb2 (최신 버전)

  
  [HISCALL].[211.175.140.137] 2024-07-23 17:07:53 (Tuesday) github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/alibaba/resources.DescribeZonesByRegion():680 - "CloudOS" : "ALIBABA", "RegionZone" : "", "ResourceType" : "REGIONZONE", "ResourceName" : "", "CloudOSAPI" : "ListZones()", "ElapsedTime" : "15.0040", "ErrorMSG" : "[SDK.TimeoutError] The request timed out 3 times(3 for retry), perhaps we should have the threshold raised a little? Connect timeout. Please set a valid ConnectTimeout.                                                    
  caused by:                                                     
  Post "http://ecs.ap-south-1.aliyuncs.com/?AcceptLanguage=en-US&AccessKeyId=LTAI5t9dGCBu8engcc3tgN6U&Action=DescribeZones&Format=JSON&RegionId=ap-south-1&Signature=WGTIVvL1tWd%2Bb7c30zk2KIbIFyk%3D&SignatureMethod=HMAC-SHA1&SignatureNonce=5a63e16371fd74f156249f6fb6569a6d&SignatureType=&SignatureVersion=1.0&Timestamp=2024-07-23T08%3A07%3A48Z&Version=2014-05-26": dial tcp 147.139.2.1:80: i/o timeout"
  [CB-SPIDER].[ERROR]: 2024-07-23 17:07:53 RegionZoneHandler.go:63, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/alibaba/resources.AlibabaRegionZoneHandler.ListRegionZone.func1() - DescribeZone failed [SDK.TimeoutError] The request timed out 3 times(3 for retry), perhaps we should have the threshold raised a little? Connect timeout. Please set a valid ConnectTimeout.
  caused by:                                                     
  Post "http://ecs.ap-south-1.aliyuncs.com/?AcceptLanguage=en-US&AccessKeyId=LTAI5t9dGCBu8engcc3tgN6U&Action=DescribeZones&Format=JSON&RegionId=ap-south-1&Signature=WGTIVvL1tWd%2Bb7c30zk2KIbIFyk%3D&SignatureMethod=HMAC-SHA1&SignatureNonce=5a63e16371fd74f156249f6fb6569a6d&SignatureType=&SignatureVersion=1.0&Timestamp=2024-07-23T08%3A07%3A48Z&Version=2014-05-26": dial tcp 147.139.2.1:80: i/o timeout 
  panic: runtime error: invalid memory address or nil pointer dereference                                                       
  [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x1f6598a]                                                      

goroutine 814 [running]:
github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/alibaba/resources.AlibabaRegionZoneHandler.ListRegionZone.func1({{0x0, 0x0}, {0xc0015f8a40, 0x1b}, {0xc0015f8a60, 0x1b}, {0xc0005f2150, 0xa}}) /home/sykim/workspace/cb-spider/cloud-control-manager/cloud-driver/drivers/alibaba/resources/RegionZoneHandler.go:65 +0x1ea created by github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/alibaba/resources.AlibabaRegionZoneHandler.ListRegionZone in goroutine 204 /home/sykim/workspace/cb-spider/cloud-control-manager/cloud-driver/drivers/alibaba/resources/RegionZoneHandler.go:53 +0x105

[powerkimhub]

@raccoon-mh @SungWoongz @MZC-CSC

• 다음 L63에서 에러가 발생 해도 계속 진행되어,
• L65에서 segmentation fault 발생하고 있습니다.
• 점검 부탁드립니다.

https://github.com/cloud-barista/cb-spider/blob/460fa14d36bb6aeb857f8477b8b9aa851ba5bd76/cloud-control-manager/cloud-driver/drivers/alibaba/resources/RegionZoneHandler.go#L63-L65

[powerkimhub]

• 점검 하시는 김에 코드내 유사 사례가 있는 지
• 빠른 점검도 부탁 드립니다.

[raccoon-mh]

Pull Request 참고 부탁드립니다..

https://github.com/cloud-barista/cb-spider/pull/1256

[powerkimhub]

• Fixed in https://github.com/cloud-barista/cb-spider/pull/1256

[powerkimhub]

@raccoon-mh

• PR 반영 후 시험 결과 다른 CSP는 이상이 없는데,
• Alibaba의 경우 여전히 아래와 같은 오류가 발생하고 있습니다.
• dial tcp 147.139.2.1:80 부분이 좀 걸리네요.
• 점검 부탁드립니다.

• 참고: 에러 로그

  [CB-SPIDER].[ERROR]: 2024-07-31 17:01:06 RegionZoneHandler.go:40, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/alibaba/resources.AlibabaRegionZoneHandler.ListRegionZone.func1() - DescribeZone failed [SDK.TimeoutError] The request timed out 3 times(3 for retry), perhaps we should have the threshold raised a little? Connect timeout. Please set a valid ConnectTimeout.
  caused by:
  Post "http://ecs.ap-south-1.aliyuncs.com/?AcceptLanguage=en-US&AccessKeyId=LTAI5t6R5fzGUqvHC9vvyvrX&Action=DescribeZones&Format=JSON&RegionId=ap-south-1&Signature=3rtBMP8iZppkJDToxBJM03ERgvs%3D&SignatureMethod=HMAC-SHA1&SignatureNonce=e57fc659312161c51ee64ee562edfb1e&SignatureType=&SignatureVersion=1.0&Timestamp=2024-07-31T08%3A01%3A01Z&Version=2014-05-26": dial tcp 147.139.2.1:80: i/o timeout 
  
  [CB-SPIDER].[ERROR]: 2024-07-31 17:01:06 RegionZoneHandler.go:71, github.com/cloud-barista/cb-spider/cloud-control-manager/cloud-driver/drivers/alibaba/resources.AlibabaRegionZoneHandler.ListRegionZone() - ListRegionZone() error : [SDK.TimeoutError] The request timed out 3 times(3 for retry), perhaps we should have the threshold raised a little? Connect timeout. Please set a valid ConnectTimeout.
  caused by:
  Post "http://ecs.ap-south-1.aliyuncs.com/?AcceptLanguage=en-US&AccessKeyId=LTAI5t6R5fzGUqvHC9vvyvrX&Action=DescribeZones&Format=JSON&RegionId=ap-south-1&Signature=3rtBMP8iZppkJDToxBJM03ERgvs%3D&SignatureMethod=HMAC-SHA1&SignatureNonce=e57fc659312161c51ee64ee562edfb1e&SignatureType=&SignatureVersion=1.0&Timestamp=2024-07-31T08%3A01%3A01Z&Version=2014-05-26": dial tcp 147.139.2.1:80: i/o timeout 
  
  [CB-SPIDER].[ERROR]: 2024-07-31 17:01:06 RegionZoneHandler.go:41, github.com/cloud-barista/cb-spider/api-runtime/common-runtime.ListRegionZone() - [SDK.TimeoutError] The request timed out 3 times(3 for retry), perhaps we should have the threshold raised a little? Connect timeout. Please set a valid ConnectTimeout.
  caused by:
  Post "http://ecs.ap-south-1.aliyuncs.com/?AcceptLanguage=en-US&AccessKeyId=LTAI5t6R5fzGUqvHC9vvyvrX&Action=DescribeZones&Format=JSON&RegionId=ap-south-1&Signature=3rtBMP8iZppkJDToxBJM03ERgvs%3D&SignatureMethod=HMAC-SHA1&SignatureNonce=e57fc659312161c51ee64ee562edfb1e&SignatureType=&SignatureVersion=1.0&Timestamp=2024-07-31T08%3A01%3A01Z&Version=2014-05-26": dial tcp 147.139.2.1:80: i/o timeout 

[powerkimhub]

@sykim-etri

• 현재 최신 버전에서는 발생하고 있지 않아서,
• 일단 close 하겠습니다.
• 재 발생시 다시 Open 부탁드립니다.

[TMI]

  발생했던 Timeout 오류 추정 중 하나는 시험 당시 Zone이 문제가 있었거나, 폐쇄 되었을 수도 있습니다.
  며칠 전에 어떤 Zone이 운영 준단된 CSP가 있었는데, 동일 CSP인지는 기억이 안나네요.
  그래서, Connection에서 해당 Zone 정보를 다른 Zone으로 교체 설정한 기억이 있습니다.
  일단, 현재는 관련 에러는 발생하고 있지 않습니다.
  유사 이슈 발생시 참고하시기 바랍니다.