search

cloudfoundry / bosh-alicloud-cpi-release

BOSH Alibaba CPI
Apache License 2.0
32 stars 20 forks source link

Bosh deployment fails: IdempotentProcessing error #44

Closed ghost closed 5 years ago

ghost commented 5 years ago

Hi,

We are getting a lot of errors like this during bosh deployment:

Task 22920

Task 22920 | 12:57:20 | Preparing deployment: Preparing deployment (00:00:20)
Task 22920 | 12:57:40 | Preparing package compilation: Finding packages to compile (00:00:00)
Task 22920 | 12:57:40 | Compiling packages: postgres-9.6.10/df1b13a169335f89748b980497594c473ce7148d
Task 22920 | 12:57:40 | Compiling packages: postgres-common/db7038137e7844ab6aa87ce80db3df0f5fd692e9
Task 22920 | 12:57:52 | Compiling packages: postgres-9.6.10/df1b13a169335f89748b980497594c473ce7148d (00:00:12)
                     L Error: CPI error 'Bosh::Clouds::CloudError' with message 'input=`{"method": "create_vm","arguments": ["c16b48fa-c484-447d-b7e2-0e158d97598c","m-gw85fkqvtvksud5muteq",{"availability_zone": "eu-central-1a","ephemeral_disk": {"size": "10_240"},"instance_type": "ecs.sn2ne.large"},{"compilation_network": {"type": "manual","ip": "10.0.3.31","netmask": "255.255.255.0","cloud_properties": {"instance_name": "bosh-compilation-ali-2-vpc-fra","internet_charge_type": "PayByBandwidth","security_group_ids": ["sg-gw8hverpzbd6h71gataq"],"vswitch_id": "vsw-gw8djpr67uu71wl5p52y3"},"default": ["dns","gateway"],"dns": ["100.100.2.136","100.100.2.138"],"gateway": "10.0.3.1"}},[],{"bosh": {"blobstores": [{"options": {"endpoint": "https://10.0.3.11:25250","password": "agent-password","tls": {"cert": {"ca": "-----BEGIN CERTIFICATE----...redacted...------END CERTIFICATE-----\n"}},"user": "agent"},"provider": "dav"}],"mbus": {"cert": {"ca": "-----BEGIN CERTIFICATE-----...redacted...-----END CERTIFICATE-----\n\n","certificate": "-----BEGIN CERTIFICATE-----...redacted...-----END CERTIFICATE-----\n","private_key": "-----BEGIN RSA PRIVATE KEY-----...redacted...PRIVATE KEY-----\n"}},"group": "bosh-ali-2-vpc-fra-cf-persistence-compilation-2f76b895-54d6-4e7e-b9c1-3afb3b915c76","groups": ["bosh-ali-2-vpc-fra","cf-persistence","compilation-2f76b895-54d6-4e7e-b9c1-3afb3b915c76","bosh-ali-2-vpc-fra-cf-persistence","cf-persistence-compilation-2f76b895-54d6-4e7e-b9c1-3afb3b915c76","bosh-ali-2-vpc-fra-sap-cf-persistence-compilation-2f76b895-54d6-4e7e-b9c1-3afb3b915c76"]}}],"context": {"director_uuid": "07e76942-2cf4-4427-b2f5-330f50af72c1","request_id": "cpi-498358"},"api_version": 1}` create instance failed with input={"Scheme":"HTTP","Method":"POST","Domain":"ecs.eu-central-1.aliyuncs.com","Port":"","RegionId":"","AcceptFormat":"JSON","QueryParams":{"AccessKeyId":"...redacted...","Action":"CreateInstance","ClientToken":"bosh-cpi-2018-12-2412:57:41.361997131+0000UTC-46e542d4-7a85-4c39","DataDisk.1.Category":"cloud_efficiency","DataDisk.1.DeleteWithInstance":"false","DataDisk.1.Encrypted":"false","DataDisk.1.Size":"20","Format":"JSON","ImageId":"m-gw85fkqvtvksud5muteq","InstanceChargeType":"PostPaid","InstanceType":"ecs.sn2ne.large","IoOptimized":"optimized","PrivateIpAddress":"10.0.3.31","RegionId":"eu-central-1","SecurityGroupId":"sg-gw8hverpzbd6h71gataq","SecurityToken":"$SECURITY_TOKEN","Signature":"...redacted...=","SignatureMethod":"HMAC-SHA1","SignatureNonce":"...redacted...","SignatureType":"","SignatureVersion":"1.0","SpotPriceLimit":"0.000000","SystemDisk.Category":"cloud_efficiency","SystemDisk.Size":"40","Timestamp":"2018-12-24T12:57:51Z","UserData":"...redacted...=","VSwitchId":"vsw-gw8djpr67uu71wl5p52y3","Version":"2014-05-26","ZoneId":"eu-central-1a"},"Headers":{"Content-Type":"application/x-www-form-urlencoded","x-sdk-client":"golang/1.0.0","x-sdk-core-version":"0.0.1","x-sdk-invoke-type":"normal"},"FormParams":{},"Content":"","OwnerId":"","ResourceOwnerAccount":"","ResourceOwnerId":"","ImageId":"m-gw85fkqvtvksud5muteq","InstanceType":"ecs.sn2ne.large","SecurityGroupId":"sg-gw8hverpzbd6h71gataq","InstanceName":"","InternetChargeType":"","AutoRenew":"","AutoRenewPeriod":"","InternetMaxBandwidthIn":"","InternetMaxBandwidthOut":"","HostName":"","Password":"","DeploymentSetId":"","ZoneId":"eu-central-1a","ClusterId":"","ClientToken":"bosh-cpi-2018-12-2412:57:41.361997131+0000UTC-46e542d4-7a85-4c39","VlanId":"","InnerIpAddress":"","SystemDiskSize":"40","SystemDiskCategory":"cloud_efficiency","SystemDiskDiskName":"","SystemDiskDescription":"","DataDisk":[{"Size":"20","SnapshotId":"","Category":"cloud_efficiency","DiskName":"","Description":"","Device":"","DeleteWithInstance":"false","Encrypted":"false"}],"NodeControllerId":"","Description":"","VSwitchId":"vsw-gw8djpr67uu71wl5p52y3","PrivateIpAddress":"10.0.3.31","IoOptimized":"optimized","OwnerAccount":"","UseAdditionalService":"","InstanceChargeType":"PostPaid","Period":"","PeriodUnit":"","Tag1Key":"","Tag2Key":"","Tag3Key":"","Tag4Key":"","Tag5Key":"","Tag1Value":"","Tag2Value":"","Tag3Value":"","Tag4Value":"","Tag5Value":"","UserData":"...redacted...=","SpotStrategy":"","KeyPairName":"","SpotPriceLimit":"0.000000","SpotInterruptionBehavior":"","RamRoleName":"","SecurityEnhancementStrategy":"","ResourceGroupId":"","HpcClusterId":"","DryRun":"","DedicatedHostId":""} : SDK.ServerError
ErrorCode: IdempotentProcessing
Recommend:
RequestId: DC8234CD-4C83-40E8-8681-E39EEA652294
Message: The previous idempotent request(s) is still processing.' in 'create_vm' CPI method (CPI request ID: 'cpi-498358')
OlegGerber commented 5 years ago

I see the same issue for last two days: {code} Task 87 | 13:12:51 | Creating missing vms: web/171ca357-14ec-4ff7-b56d-416064e37c1e (0) (00:00:12) L Error: CPI error 'Bosh::Clouds::CloudError' with message 'input=`{"method": "create_vm SDK.ServerError ErrorCode: IdempotentProcessing Recommend: RequestId: 46F1EE0D-ACF5-46E5-B8FF-6706ACA60E9C Message: The previous idempotent request(s) is still processing.' in 'create_vm' CPI method (CPI request ID: 'cpi-419983') {code}

xiaozhu36 commented 5 years ago

Fixed by release v20