Failed to run "Getting Started with Kubernetes" on minikube

Haufe-Lexware / wicked.haufe.io

An API Management system based on Mashape Kong

http://wicked.haufe.io

Other

124 stars 37 forks source link

Failed to run "Getting Started with Kubernetes" on minikube #139

Open ghost opened 5 years ago

ghost commented 5 years ago

i am trying to follow the steps specified in (https://wicked.haufe.io/gettingstarted-k8s.html)

after downloading the wicked release version 1.0.0.beta11 with curl command specified on the tutorial, when i run the install command: helm install --set minikubeIP=$(minikube ip) --name test wicked

the command can't start pods successfully:

NAME	READY	STATUS	RESTARTS	AGE
test-wicked-api-9d46b957f-7qsnz	0/1	CrashLoopBackOff	6	9m
test-wicked-auth-7cc965cf9c-vldnr	0/1	Init:CrashLoopBackOff	6	9m
test-wicked-kong-86688b887-s589n	0/1	Init:CrashLoopBackOff	6	9m
test-wicked-kong-database-54f9c7d5bb-pg8tv	1/1	Running	0	9m
test-wicked-kong-adapter-58885479c7-6mtjv	0/1	Init:CrashLoopBackOff	6	9m
test-wicked-portal-768888c99d-9h7qg	0/1	Init:CrashLoopBackOff	6	9m
test-wicked-redis-c9d49995c-br49j	1/1	Running	0	9m

the logs for wicked-api shows:

debug: [+   0ms] portal-api:utils               loadGlobals()
debug: [+   1ms] portal-api:dao:pg:utils        resolveDatabase(wicked) resolves to: wicked
debug: [+   0ms] portal-api:dao:pg:utils        {"host":"wicked-database","port":"5432","user":"kong","password":"kong","database":"wicked","max":10,"connectionTimeoutMillis":10000}
debug: [+   0ms] portal-api:dao:pg:utils        getPoolOrClient: Trying to connect
debug: [+  24ms] portal-api:dao:pg:utils        getPoolOrClient: Connect to wicked database failed.
{ Error: getaddrinfo ENOTFOUND wicked-database wicked-database:5432
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:57:26)
  errno: 'ENOTFOUND',
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'wicked-database',
  host: 'wicked-database',
  port: 5432 }
debug: [+   3ms] portal-api:dao:pg:utils        {"errno":"ENOTFOUND","code":"ENOTFOUND","syscall":"getaddrinfo","hostname":"wicked-database","host":"wicked-database","port":5432}
debug: [+   0ms] portal-api:dao:pg:utils        getPoolOrClient: pool.connect returned an unknown/unexpected error; error code: ENOTFOUND
{ Error: getaddrinfo ENOTFOUND wicked-database wicked-database:5432
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:57:26)
  errno: 'ENOTFOUND',
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'wicked-database',
  host: 'wicked-database',
  port: 5432 }
error: [+   2ms] portal-api:initializer         {"errno":"ENOTFOUND","code":"ENOTFOUND","syscall":"getaddrinfo","hostname":"wicked-database","host":"wicked-database","port":5432}
error: [+   1ms] portal-api:server              initializer.checkDynamicConfig() failed!
error: [+   0ms] portal-api:server              {"errno":"ENOTFOUND","code":"ENOTFOUND","syscall":"getaddrinfo","hostname":"wicked-database","host":"wicked-database","port":5432}
{ Error: getaddrinfo ENOTFOUND wicked-database wicked-database:5432
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:57:26)
  errno: 'ENOTFOUND',
  code: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'wicked-database',
  host: 'wicked-database',
  port: 5432 }
error: [+   0ms] portal-api:server              Error: getaddrinfo ENOTFOUND wicked-database wicked-database:5432
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:57:26)

System Information:

$ minikube version
minikube version: v0.30.0

$ helm version
Client: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.14.1
BuildVersion:   18B75

$ vboxmanage --version
5.2.20r125813

DonMartin76 commented 5 years ago

Will check first thing in the morning tomorrow.

marcelo carvalho notifications@github.com schrieb am Mo. 19. Nov. 2018 um 21:21:

i am trying to follow the steps specified in ( https://wicked.haufe.io/gettingstarted-k8s.html)

after downloading the wicked release version 1.0.0.beta11 with curl command specified on the tutorial, when i run the install command: helm install --set minikubeIP=$(minikube ip) --name test wicked

the command can't start pods successfully: NAME READY STATUS RESTARTS AGE test-wicked-api-9d46b957f-7qsnz 0/1 CrashLoopBackOff 6 9m test-wicked-auth-7cc965cf9c-vldnr 0/1 Init:CrashLoopBackOff 6 9m test-wicked-kong-86688b887-s589n 0/1 Init:CrashLoopBackOff 6 9m test-wicked-kong-database-54f9c7d5bb-pg8tv 1/1 Running 0 9m test-wicked-kong-adapter-58885479c7-6mtjv 0/1 Init:CrashLoopBackOff 6 9m test-wicked-portal-768888c99d-9h7qg 0/1 Init:CrashLoopBackOff 6 9m test-wicked-redis-c9d49995c-br49j 1/1 Running 0 9m

the logs for wicked-api shows:

debug: [+ 0ms] portal-api:utils loadGlobals() debug: [+ 1ms] portal-api:dao:pg:utils resolveDatabase(wicked) resolves to: wicked debug: [+ 0ms] portal-api:dao:pg:utils {"host":"wicked-database","port":"5432","user":"kong","password":"kong","database":"wicked","max":10,"connectionTimeoutMillis":10000} debug: [+ 0ms] portal-api:dao:pg:utils getPoolOrClient: Trying to connect debug: [+ 24ms] portal-api:dao:pg:utils getPoolOrClient: Connect to wicked database failed. { Error: getaddrinfo ENOTFOUND wicked-database wicked-database:5432 at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:57:26) errno: 'ENOTFOUND', code: 'ENOTFOUND', syscall: 'getaddrinfo', hostname: 'wicked-database', host: 'wicked-database', port: 5432 } debug: [+ 3ms] portal-api:dao:pg:utils {"errno":"ENOTFOUND","code":"ENOTFOUND","syscall":"getaddrinfo","hostname":"wicked-database","host":"wicked-database","port":5432} debug: [+ 0ms] portal-api:dao:pg:utils getPoolOrClient: pool.connect returned an unknown/unexpected error; error code: ENOTFOUND { Error: getaddrinfo ENOTFOUND wicked-database wicked-database:5432 at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:57:26) errno: 'ENOTFOUND', code: 'ENOTFOUND', syscall: 'getaddrinfo', hostname: 'wicked-database', host: 'wicked-database', port: 5432 } error: [+ 2ms] portal-api:initializer {"errno":"ENOTFOUND","code":"ENOTFOUND","syscall":"getaddrinfo","hostname":"wicked-database","host":"wicked-database","port":5432} error: [+ 1ms] portal-api:server initializer.checkDynamicConfig() failed! error: [+ 0ms] portal-api:server {"errno":"ENOTFOUND","code":"ENOTFOUND","syscall":"getaddrinfo","hostname":"wicked-database","host":"wicked-database","port":5432} { Error: getaddrinfo ENOTFOUND wicked-database wicked-database:5432 at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:57:26) errno: 'ENOTFOUND', code: 'ENOTFOUND', syscall: 'getaddrinfo', hostname: 'wicked-database', host: 'wicked-database', port: 5432 } error: [+ 0ms] portal-api:server Error: getaddrinfo ENOTFOUND wicked-database wicked-database:5432 at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:57:26)

System Information:

$ minikube version minikube version: v0.30.0

$ helm version Client: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"} Server: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}

$ sw_vers ProductName: Mac OS X ProductVersion: 10.14.1 BuildVersion: 18B75

$ vboxmanage --version 5.2.20r125813

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Haufe-Lexware/wicked.haufe.io/issues/139, or mute the thread https://github.com/notifications/unsubscribe-auth/AG_Mo9H436pwusULejt0m2IHjiVJ6Azcks5uwxKsgaJpZM4Yp1Af .

DonMartin76 commented 5 years ago

The URL for the Postgres instance is not correct; the question is why. The rest of the crashes are just consequences. I will try to reproduce it.

DonMartin76 commented 5 years ago

I can reproduce. I am still puzzled though. Will come back with a solution (hopefully) later.

DonMartin76 commented 5 years ago

OK, so, two things which are both due to "not quite finished" documentation, and please correct me if I am off on these things:

Newer versions of minikube automatically deploy Kubernetes with RBAC turned on. This makes it necessary to install wicked with --set useRbac=true (missing in the documentation)
Did you create your own static configuration and check it in to a repository which you then pointed to with the configuration (using --set config.repository=https://... and --set config.credentials=<username>:<password>)?
The default wicked-sample-config is not yet adapted to wicked 1.0.0, and thus will not work (some things are missing, such as the Postgres configuration). This is noted in a sentence in the "How to Beta" documentation: "It is also mandatory that you have checked in the API configuration into a repository which is accessible from within the Kubernetes cluster." That could be more explicit... ;-)

I will take this as a nudge to update the wicked-sample-config to wicked 1.0.0, but I wanted to get the RC out first (due in the next weeks).

Please check if this helped.

DonMartin76 commented 5 years ago

To make it more clear: The main issue here is the outdated wicked-sample-config repository.

stackedsax commented 5 years ago

@DonMartin76 I'm not so sure it's to do with the wicked-sample-config repo. It seems to me that helm is not getting nearly that far.

To me, it looks like the wicked.portal-api image is not using the k8s environment variable and therefore not loading the values in node_modules/portal-env/initial-config/static/env/k8s.json.

Instead, it looks like the image is trying to boot up using the values in node_modules/portal-env/initial-config/static/env/default.json. Specifically, line 85: "value": "wicked-database".

envName is indeed set in the values.yaml, and I just tried setting it explicitly with --set config.envName=k8s but to no avail. Even without the --set config.envName I can see that the output of a helm --dry-run --debug shows config:envName:k8s as a computed value.

My guess at the moment is that something in node_modules/portal-env/src/config-updater.js isn't working right. The config-updater.js that is on the image looks like a newer version than is checked into any branch at https://github.com/apim-haufe-io/wicked.portal-env so I'm going to stop digging for the moment. Are there changes that could be checked in?

Any suggestions for where else to poke?

stackedsax commented 5 years ago

Looking at the debug logs, it looks like PORTAL_STORAGE_PGHOST is being set incorrectly. It looks like line 285 of the updated (but not yet checked in?) version of config-updater.js:

targetDefaultEnv.PORTAL_STORAGE_PGHOST = sourceDefaultEnv.PORTAL_STORAGE_PGHOST;

I don't totally understand the logic behind the targetConfig and targetDefaultEnv variables, but to me lines 281 and 282 look suspicously similar:

        const sourceDefaultEnv = loadEnv(sourceConfig, 'default');                                  
        const targetDefaultEnv = loadEnv(targetConfig, 'default');

If I understand things correctly, I want that 'default' next to targetCofig to be 'k8s' when I run helm.

stackedsax commented 5 years ago

@DonMartin76 @mcarvalho

Took another look at my last comment this morning, and I think those two lines 281 and 282 in updateStep14_v1_0_0e are fine.

However, I noticed in updateStep12_v1_0_0c you do this:

    // Also for k8s env                                                                                                                                                       
    const sourceK8sEnv = loadEnv(sourceConfig, 'k8s');                                                                                                                        
    if (existsEnv(targetConfig, 'k8s')) {                                                                                                                                     
        const targetK8sEnv = loadEnv(targetConfig, 'k8s');                                                                                                                    
        if (updateEnv(sourceK8sEnv, targetK8sEnv))                                                                                                                            
            saveEnv(targetConfig, 'k8s', targetK8sEnv);                                                                                                                       
    }

And in updateStep10_v1_0_0a you do this:

    // Also for k8s env                                                                                                                                                       
    const sourceK8sEnv = loadEnv(sourceConfig, 'k8s');                                                                                                                        
    if (existsEnv(targetConfig, 'k8s')) {                                                                                                                                     
        const targetK8sEnv = loadEnv(targetConfig, 'k8s');                                                                                                                    
        if (updateEnv(sourceK8sEnv, targetK8sEnv))                                                                                                                            
            saveEnv(targetConfig, 'k8s', targetK8sEnv);                                                                                                                       
    } else {                                                                                                                                                                  
        // Does not yet exist, just copy it                                                                                                                                   
        saveEnv(targetConfig, 'k8s', sourceK8sEnv);                                                                                                                           
    }

So I thought something similar needs to be added in updateStep14_v1_0_0e. The PR linked above is my first stab at what I think needs to be added to update the PORTAL_STORAGE_PGHOST variable.

Oh, and I found the checked in code in the wicked_1_0 branch. I don't know how I missed that obviously-named branch yesterday.

DonMartin76 commented 5 years ago

Looks very good so far, thanks a lot for your efforts! Have you seen other issues of this type? I appreciate it a lot that you're taking a stab at those - once you get everything up and running, operating wicked is usually smooth, but the configuration is sometimes tricky.

stackedsax commented 5 years ago

I've switched to trying to get the 12.1 helm install working, but once I get that going I'll come back to this.

I did feel that the config-updater.js felt like an incredible amount of code for what it's doing. Surely there are configuration frameworks which can handle the mass of boilerplate code and simplify a lot of what you're doing here?

cuonglm commented 5 years ago

@DonMartin76 I try with v0.12.1 as documentation, it doesn't work. API container failed with this error:

Setting owner of /var/portal-api to wicked:wicked
Granting read/write rights to user 'wicked' for Swagger files...
Starting API, running as user 'wicked'...
info: [+  --ms] portal-env:logger              Using plain logging format on non-containerized OS; override with LOG_PLAIN=false
info: [+  --ms] portal-env:logger              Setting up logging with log level "debug" (override with LOG_LEVEL)
info: [+ NaNms] portal-api:dao:pg:utils        initPrometheus()
info: [+ 101ms] portal-api:principal           Creating principal ID for this instance: fc4089780c84c20bf98171479265d837c2aa7b6e
debug: [+  99ms] portal-api:utils               init()
debug: [+   1ms] portal-api:app                 Configuring logger.
info: [+ 390ms] portal-api:server              Portal API environment: default
info: [+   6ms] portal-api:server              Running as user 'wicked'
debug: [+   0ms] portal-env:env-reader          resolveStaticConfig():
debug: [+   0ms] portal-env:env-reader          checkStaticConfigDir(/var/portal-api/static)
debug: [+   0ms] portal-env:env-reader          Found file globals.json: /var/portal-api/static/globals.json
debug: [+   0ms] portal-env:env-reader          Resolved config dir: /var/portal-api/static
debug: [+   0ms] portal-env:env-reader          /var/portal-api/static
debug: [+   0ms] portal-env:env-reader          checkEnvironment() - /var/portal-api/static, env: default
info: [+   1ms] portal-env:env-reader          Reading config from: /var/portal-api/static
debug: [+   0ms] portal-env:env-reader          getLocalIPs()
debug: [+   0ms] portal-env:env-reader          ["10.244.5.100"]
debug: [+   0ms] portal-env:env-reader          Setting LOCAL_API_URL: http://10.244.5.100:3001
debug: [+   0ms] portal-env:env-reader          loadEnvironment() - /var/portal-api/static
debug: [+   1ms] portal-env:env-reader          Environment variable HELM_NAME is already set to "acb-staging-wicked". Skipping in this configuration.
debug: [+   0ms] portal-env:env-reader          Environment variable K8S_NAMESPACE is already set to "default". Skipping in this configuration.
debug: [+   1ms] portal-env:env-reader          Environment variable PORTAL_INITIALUSERS0_PASSWORD is already set to "wicked". Skipping in this configuration.
debug: [+   0ms] portal-env:env-reader          Environment variable PORTAL_NETWORK_APIHOST is already set to "api.portal.acb.brank.as". Skipping in this configuration.
debug: [+   0ms] portal-env:env-reader          Environment variable PORTAL_NETWORK_PORTALHOST is already set to "portal.acb.braank.as". Skipping in this configuration.
debug: [+   0ms] portal-env:env-reader          Environment variable PORTAL_SESSIONSTORE_TYPE is already set to "redis". Skipping in this configuration.
debug: [+   1ms] portal-env:env-reader          Replacing PORTAL_CONFIG_BASE with "/override/this/" in "${PORTAL_CONFIG_BASE}/dynamic".
debug: [+   0ms] portal-env:env-reader          Replacing PORTAL_CONFIG_BASE with "/override/this/" in "${PORTAL_CONFIG_BASE}/static".
/usr/src/app/node_modules/portal-env/env-reader.js:118
        throw new Error('portal-env:checkEnvironment() - The environment variable PORTAL_API_STATIC_CONFIG does not match the resolved configuration path. Please change the environment files, or pass in the correct path to the static configuration using a pre-set environment variable PORTAL_API_STATIC_CONFIG: ' + process.env.PORTAL_API_STATIC_CONFIG + ' vs. ' + staticConfigPath);
        ^

Error: portal-env:checkEnvironment() - The environment variable PORTAL_API_STATIC_CONFIG does not match the resolved configuration path. Please change the environment files, or pass in the correct path to the static configuration using a pre-set environment variable PORTAL_API_STATIC_CONFIG: /override/this/static vs. /var/portal-api/static
    at Function.envReader.checkEnvironment (/usr/src/app/node_modules/portal-env/env-reader.js:118:15)
    at Object.<anonymous> (/usr/src/app/bin/api:49:11)
    at Module._compile (internal/modules/cjs/loader.js:688:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
    at Module.load (internal/modules/cjs/loader.js:598:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:537:12)
    at Function.Module._load (internal/modules/cjs/loader.js:529:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:741:12)
    at startup (internal/bootstrap/node.js:285:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:739:3)

Other pods in Init:Error state.

Does https://github.com/apim-haufe-io/wicked-sample-config work with 0.x.x version?

DonMartin76 commented 5 years ago

Possibly not anymore. I recommend creating a new configuration via the kickstarter, or use the beta version of wicked 1.0.0 (even more recommended). There may still be some hiccups there, but I hope to be able to close those in the course of the next couple of days.

DonMartin76 commented 5 years ago

The recommended local deployment method will be the wicked-cli as of the soon-to-be-release 1.0.0-rc.1 release. More information to come.