search

nginxinc / kic-reference-architectures

MARA: Modern Application Reference Architecture
Apache License 2.0
634 stars 100 forks source link

Error pulumi install #191

Open efcunha opened 2 years ago

efcunha commented 2 years ago

Describe the bug Error installing: kubernetes:core/v1:ConfigMap

Your environment $ pulumi version v3.38.0

$ node --version v18.8.0

$ python --version Python 3.9.13

KIC v1.0.1

Ubuntu 22.04

$ cat Pulumi.devops.yaml

config: aws:profile: default aws:region: us-west-2 kubernetes:infra_type: AWS

To Reproduce

$ ./start.sh

Updating (devops)

View Live: https://app.pulumi.com/efcunha/aws-eks/devops/updates/6

 Type                                   Name                                    Status       Info

Diagnostics: kubernetes:core/v1:ConfigMap (aws-eks-devops-nodeAccess): error: failed to initialize discovery client: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1alpha1"

pulumi:pulumi:Stack (aws-eks-devops): aws default profile vpc id: vpc-025729a9f6169ec49 warning: aws:ec2/getSubnetIds:getSubnetIds verification warning: Deprecated Resource public subnets: ['subnet-0064d171ae22441ae', 'subnet-0370ad0da9abcc517', 'subnet-0316b175509d86c3d', 'subnet-0278bf3e9620b24b5'] warning: aws:ec2/getSubnetIds:getSubnetIds verification warning: Deprecated Resource public subnets: ['subnet-0bab25610049dbc2f', 'subnet-0143a10ed96adf3cd', 'subnet-02082be33e689f992', 'subnet-033cc97107714b619'] error: Resource monitor has terminated, shutting down

Warning: spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[0].key: beta.kubernetes.io/os is deprecated since v1.14; use "kubernetes.io/os" instead
Warning: spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[0].matchExpressions[1].key: beta.kubernetes.io/arch is deprecated since v1.14; use "kubernetes.io/arch" instead

Resources:

Duration: 10m47s

(base) efcunha@devops:~/GitHub/kic-reference-architectures$ ...

qdzlug commented 2 years ago

HI @efcunha - please try again with the latest updates. One of the changes in the last PR addresses this issue. Note that this PR changes the way you start the MARA process - ./pulumi/python/runner - the start.sh script is deprecated and only used for workstation based installs now. The script will tell you this as well.

Let me know if you have issues.

Jay

efcunha commented 2 years ago

~/kic-reference-architectures/pulumi/python$ ./runner -s devops -p aws validate stack configuration file [/home/efcunha/GitHub/kic-reference-architectures/config/pulumi/Pulumi.devops.yaml] does not exist creating new configuration based on user input AWS region to use [us-east-1]: us-west-2 AWS region: us-west-2 AWS profile to use [none] (enter "none" for none): default AWS profile: default AWS availability zones to use with VPC [us-west-2a, us-west-2b, us-west-2c, us-west-2d] (separate with commas): AWS availability zones: us-west-2a, us-west-2b, us-west-2c, us-west-2d EKS Kubernetes version [1.21]: 1.23 EKS Kubernetes version: 1.23 EKS instance type [t2.large]: EKS instance type: t2.large Minimum number compute instances for EKS cluster [3]: EKS minimum cluster size: 3 Maximum number compute instances for EKS cluster [12]: EKS maximum cluster size: 12 Desired number compute instances for EKS cluster [3]: EKS maximum cluster size: 3 Prometheus administrator password: Bank of Sirius Accounts Database password: Bank of Sirius Ledger Database password: Bank of Sirius demo site login username [testuser]: Bank of Sirius demo site login password [password]:

$ cat /kic-reference-architecturesconfig/pulumi/Pulumi-devops.yaml

config: aws:profile: default aws:region: us-west-2 eks:desired_capacity: 3 eks:instance_type: t2.large eks:k8s_version: '1.23' eks:max_size: 12 eks:min_size: 3 kubernetes:infra_type: AWS vpc:azs:

kic-reference-architectures/pulumi/python$ ./runner -s devops -p aws up

` Duration: 11m49s

ERROR:root:Error running Pulumi operation [up] with provider [aws] for stack [devops] Traceback (most recent call last): File "/home/efcunha/GitHub/kic-reference-architectures/pulumi/python/automation/main.py", line 562, in main() File "/home/efcunha/GitHub/kic-reference-architectures/pulumi/python/automation/main.py", line 277, in main raise e File "/home/efcunha/GitHub/kic-reference-architectures/pulumi/python/automation/main.py", line 273, in main pulumi_cmd(provider=provider, env_config=env_config) File "/home/efcunha/GitHub/kic-reference-architectures/pulumi/python/automation/main.py", line 538, in up pulumi_project.on_success(params) File "/home/efcunha/GitHub/kic-reference-architectures/pulumi/python/automation/providers/aws.py", line 206, in _update_kubeconfig res, err = external_process.run(cmd) File "/home/efcunha/GitHub/kic-reference-architectures/pulumi/python/venv/lib/python3.9/site-packages/kic_util/external_process.py", line 23, in run raise ExternalProcessExecError(msg, cmd) kic_util.external_process.ExternalProcessExecError: aws --region us-west-2 --profile default eks update-kubeconfig --name aws-eks-devops-eksCluster-719a8f4 when running: Failed to execute external process: aws --region us-west-2 --profile default eks update-kubeconfig --name aws-eks-devops-eksCluster-719a8f4

Error: 'NoneType' object is not iterable `

qdzlug commented 2 years ago

Hi @efcunha,

Do you have multiple entries in your kubeconfig file, or do you have included directories with additional kubeconfig files? This error - NoneType object is not iterable - is something I saw in the past when the AWS K8 configuration was added to an existing kubeconfig. If this is the case, could you try with an empty kubeconfig file (~/.kube/config) and aKUBECONFIGenv variable that just points to the standard config (ie,KUBECONFIG=~/.kube/config`)

If this is not the case, let me know and I can look deeper into it; I just tested myself and this should work on AWS (it's passing both our deployment tests, and I deployed it locally as well).

Cheers,

Jay

efcunha commented 2 years ago

$ cat /kic-reference-architecturesconfig/pulumi/Pulumi-devops.yaml

config: aws:profile: default aws:region: us-west-2 eks:desired_capacity: 3eks:k8s_version: '1.23' eks:instance_type: t2.large eks:k8s_version: '1.23' eks:max_size: 12 eks:min_size: 3 kubernetes:infra_type: AWS vpc:azs:

us-west-2a us-west-2b us-west-2c us-west-2d kic-reference-architectures/pulumi/python$ ./runner -s devops -p aws up

Diagnostics: kubernetes:helm.sh/v3:Release (elastic): warning: Helm release "elastic" was created but has a failed status. Use the helm command to investigate the error, correct it, then retry. Reason: timed out waiting for the condition error: 1 error occurred:

Resources:

Duration: 6m9s

stderr:

qdzlug commented 2 years ago

@efcunha - thanks for the output; running some tests now.

qdzlug commented 2 years ago

@efcunha - thanks for your patience. That error above - where logstore (elasticsearch) times out is tied to how long elasticsearch takes to become available. By default, pulumi waits for 5 minutes and then bails out. This is normally long enough, but in my testing here I see that it's taking about 7 minutes for it to become available.

There is, fortunately, a tunable for this - you can set it via 

pulumi -C ./pulumi/python/kubernetes/logstore config set logstore:helm_timeout=600

This value is in seconds, and I normally put it at 600 (10 mins) which is usually sufficient.

Please give it a try with this set and let me know how it goes - you will need to destroy the existing deployment, since helm leaves things in a weird state when it fails like this.

Cheers,

Jay

efcunha commented 2 years ago

Hello, no problem, I like to keep testing, news for me is a preazer.

falyoun commented 1 year ago

Same here