aws-quickstart / quickstart-ibm-icp-for-data

AWS Quick Start Team
Apache License 2.0
14 stars 19 forks source link

Issues deployment of cp4d with quickstart #43

Open sandeepbhootna opened 3 years ago

sandeepbhootna commented 3 years ago

Hi Team,

Could you please have a look into the issue only error (Installation of assembly lite Failed) I can see in the bootstrap is as below, We are using quickstart deployment (CloudFormation) in aws for single AZ. If you need more info please do let us know.

-------------Installing operator group------------- operatorgroup.operators.coreos.com/ibm-cp-data-operator-group created catalogsource.operators.coreos.com/ibm-cp-data-operator-catalog created Wait for catalog installation to complete subscription.operators.coreos.com/ibm-cp-data-operator-subscription created Wait for subscription to complete [✓] CASE launch script completed successfully OK

[21/05/17 19:15:32.149072 UTC] 7f4cde87b740 I main(305) installOperator : Execute install operator returned Running

[21/05/17 19:15:32.440071 UTC] 7f4cde87b740 I main(199) installCPD : Create new project with user defined project name zen,retcode=0 [21/05/17 19:15:32.440312 UTC] 7f4cde87b740 I main(216) installCPD : Start installing Lite package [21/05/17 19:15:32.442464 UTC] 7f4cde87b740 I main(358) installAssemblies : Execute install command for assembly lite [21/05/17 19:15:32.945790 UTC] 7f4cde87b740 I main(362) installAssemblies : Execute install command for assembly lite returned 0 [21/05/17 19:16:33.200335 UTC] 7f4cde87b740 I main(369) installAssemblies : Get install status for assembly lite is Installing

[21/05/17 19:17:33.495643 UTC] 7f4cde87b740 I main(369) installAssemblies : Get install status for assembly lite is Failed

[21/05/17 19:17:33.495827 UTC] 7f4cde87b740 E main(371) installAssemblies : Installation of assembly lite Failed [21/05/17 19:17:33.495901 UTC] 7f4cde87b740 E main(1290) main : Exception with message Installation of assembly lite Failed #######################################

I think you must require the parameters of the stack for investigation, here you go..

{ "Stacks": [ { "StackId": "arn:aws:cloudformation:eu-west-1:339493409635:stack/VQD-IBM-Cloud-Pak-for-Data-3/13aeb380-b738-11eb-b955-028ffd2cfbdb", "DriftInformation": { "StackDriftStatus": "NOT_CHECKED" }, "Description": "Root template for an IBM Cloud Pak for Data deployment. This is the root template for a collection of nested stacks that make up the full CloudPak for Data deployment. WARNING This template creates EC2 instances and related resources. You will be billed for the AWS resources used if you create a stack from this template. (qs-1rddjo02q)", "Parameters": [ { "ParameterValue": "I agree", "ParameterKey": "LicenseAgreement" }, { "ParameterValue": "10.0.160.0/20", "ParameterKey": "PublicSubnet3CIDR" }, { "ParameterValue": "True", "ParameterKey": "CDE" }, { "ParameterValue": "10.0.32.0/19", "ParameterKey": "PrivateSubnet2CIDR" }, { "ParameterValue": "0.0.0.0/0", "ParameterKey": "BootNodeAccessCIDR" }, { "ParameterValue": "1", "ParameterKey": "NumberOfAZs" }, { "ParameterValue": "quickstart-ibm-icp-for-data/", "ParameterKey": "QSS3KeyPrefix" }, { "ParameterValue": "m5.xlarge", "ParameterKey": "MasterInstanceType" }, { "ParameterValue": "3", "ParameterKey": "NumberOfCompute" }, { "ParameterValue": "True", "ParameterKey": "DV" }, { "ParameterValue": "10.0.64.0/19", "ParameterKey": "PrivateSubnet3CIDR" }, { "ParameterValue": "10.0.0.0/16", "ParameterKey": "VPCCIDR" }, { "ParameterValue": "External", "ParameterKey": "PrivateCluster" }, { "ParameterValue": "vqd-pole-openshift-cluster", "ParameterKey": "ClusterName" }, { "ParameterValue": "10.0.128.0/20", "ParameterKey": "PublicSubnet1CIDR" }, { "ParameterValue": "", "ParameterKey": "AdminPassword" }, { "ParameterValue": "m4.4xlarge", "ParameterKey": "OCSInstanceType" }, { "ParameterValue": "3", "ParameterKey": "NumberOfMaster" }, { "ParameterValue": "cp4d-test", "ParameterKey": "KeyPairName" }, { "ParameterValue": "aws-quickstart", "ParameterKey": "QSS3BucketName" }, { "ParameterValue": "3", "ParameterKey": "NumberOfOCS" }, { "ParameterValue": "OCS", "ParameterKey": "StorageType" }, { "ParameterValue": "vqdpolesoultion.co.uk", "ParameterKey": "DomainName" }, { "ParameterValue": "3.5.2", "ParameterKey": "ICPDVersion" }, { "ParameterValue": "eu-west-1a", "ParameterKey": "AvailabilityZones" }, { "ParameterValue": "False", "ParameterKey": "EnableFips" }, { "ParameterValue": "True", "ParameterKey": "OpenScale" }, { "ParameterValue": "s3://vqd-polesoultion-bucket-1/pull-secret.txt", "ParameterKey": "RedhatPullSecret" }, { "ParameterValue": "True", "ParameterKey": "WKC" }, { "ParameterValue": "zen", "ParameterKey": "Namespace" }, { "ParameterValue": "m5.4xlarge", "ParameterKey": "ComputeInstanceType" }, { "ParameterValue": "10.0.0.0/19", "ParameterKey": "PrivateSubnet1CIDR" }, { "ParameterValue": "True", "ParameterKey": "WSL" }, { "ParameterValue": "", "ParameterKey": "APIKey" }, { "ParameterValue": "10.0.144.0/20", "ParameterKey": "PublicSubnet2CIDR" }, { "ParameterValue": "", "ParameterKey": "PortworxSpec" }, { "ParameterValue": "True", "ParameterKey": "WML" }, { "ParameterValue": "us-east-1", "ParameterKey": "QSS3BucketRegion" }, { "ParameterValue": "True", "ParameterKey": "Spark" }, { "ParameterValue": "cp", "ParameterKey": "APIUsername" }, { "ParameterValue": "10.128.0.0/14", "ParameterKey": "ClusterNetworkCIDR" }, { "ParameterValue": "vqd-polesoultion-bucket-1", "ParameterKey": "ICPDDeploymentLogsBucketName" } ], "EnableTerminationProtection": false, "CreationTime": "2021-05-17T17:48:24.665Z", "Capabilities": [ "CAPABILITY_NAMED_IAM", "CAPABILITY_AUTO_EXPAND" ], "StackName": "VQD-IBM-Cloud-Pak-for-Data-3", "NotificationARNs": [], "StackStatus": "ROLLBACK_COMPLETE", "DisableRollback": false, "RollbackConfiguration": { "RollbackTriggers": [] }, "DeletionTime": "2021-05-17T19:20:06.038Z" } ] }

Help really appreciated..

Thank you, Sandeep

sandeepbhootna commented 3 years ago

Can you please look into this.. Although it says check the logs but post_install is not crated, I can provide icpd_install.log

WaitCondition received failed message: 'FAILURE: Check logs in S3 log bucket or on the Boot node EC2 instance in /ibm/logs/icpd_install.log and /ibm/logs/post_install.log' for uniqueId: arn:aws:cloudformation:eu-west-1:339493409635:stack/VQD-IBM-Cloud-Pak-for-Data-4-CloudPakDataStack-VTDKT0ZQ27YQ/6daec1d0-bef0-11eb-bf80-0231dda8da6b

Embedded stack arn:aws:cloudformation:eu-west-1:339493409635:stack/VQD-IBM-Cloud-Pak-for-Data-4-CloudPakDataStack-VTDKT0ZQ27YQ/6daec1d0-bef0-11eb-bf80-0231dda8da6b was not successfully created: The following resource(s) failed to create: [ICPDInstallationCompleted].

sandeepbhootna commented 3 years ago

icpd_install (1).log

shaithal commented 3 years ago

can you share output of oc get pods -n zen oc logs -f $(oc get pods -n zen | grep 'cpd-install' | awk '{print $1}') oc get pods -n cpd-meta-ops

sandeepbhootna commented 3 years ago

I am using below template

https://aws-quickstart.s3.amazonaws.com/quickstart-ibm-icp-for-data/templates/ibm-cloudpak-root.template.yaml

After couple of hours it just rolled back complete environment.

parthakom2 commented 3 years ago

@sandeepbhootna can you please retry with disabling Rollback-on-failure option. The lite assembly could fail for multiple reasons like incorrect container registry APIKey or a storage issue.

sandeepbhootna commented 3 years ago

@parthakom2, What about timeout field, Do I need to leave blank this field?

sandeepbhootna commented 3 years ago

Output oc get pods -n zen
(No resource found in zen namespace) oc logs -f $(oc get pods -n zen | grep 'cpd-install' | awk '{print $1}') As first is failed, nothing is coming No output oc get pods -n cpd-meta-ops 3 pods can be seen 2 are running, (ibm-cp-data-operator-) and (meta-api-deploy--blctx) 1 is completed, setup-job-cknxc Ready(0/1)

shaithal commented 3 years ago

oc logs -f ibm-cp-data-operator-xxx get logs of this pod.

sandeepbhootna commented 3 years ago

Hello Sharath,

Please find below screen shot, as there is error to find out the logs .. pod not found

@.***D75701.7F7CFB40]

Regards, Sandeep From: Sharath Aithal @.> Sent: 01 June 2021 15:57 To: aws-quickstart/quickstart-ibm-icp-for-data @.> Cc: Sandeep Bhootna @.>; Mention @.> Subject: Re: [aws-quickstart/quickstart-ibm-icp-for-data] Issues deployment of cp4d with quickstart (#43)

oc logs -f ibm-cp-data-operator-xxx get logs of this pod.

- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Faws-quickstart%2Fquickstart-ibm-icp-for-data%2Fissues%2F43%23issuecomment-852193562&data=04%7C01%7CSandeep.Bhootna%40viqtordavis.com%7C4e2617b94a8347a913cb08d9250d7329%7C0f5a504090784bb898ef123b88b65cc8%7C1%7C0%7C637581561965497311%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=coWpva9ApsBCbZVz0%2BjzeUf88gXLYx%2F%2Fvk4pLw6uWZE%3D&reserved=0, or unsubscribehttps://gbr01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAT7DROTARSACEC5UCHKYBTTTQTYKDANCNFSM45BD5C2A&data=04%7C01%7CSandeep.Bhootna%40viqtordavis.com%7C4e2617b94a8347a913cb08d9250d7329%7C0f5a504090784bb898ef123b88b65cc8%7C1%7C0%7C637581561965507302%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=7YQwT246G7BJmgzZtNgNG18QtSr99Eg9hftPB4x6y4w%3D&reserved=0.

sandeepbhootna commented 3 years ago

Deploying cp4d again, it is in create in progress for more than 5 hours