jbossdemocentral / coolstore-microservice

CoolStore is a containerised polyglot microservices application consisting of services based on JBoss, NodeJS, Spring Boot, WildFly Swarm, Vert.x, Netflix OSS and more running on OpenShift
299 stars 324 forks source link

"Nexus still not ready, I GIVE UP!" - plain setup on 32 GByte, 8-Core + SSD Fedora 25 Notebook #48

Open karstengresch opened 7 years ago

karstengresch commented 7 years ago

Hi all,

tried to get the coolstore demo running (latest version tried: 9dc4d65f67d573538b86bc46952a5a548b4e3253) since several days, assuming that the documentation in https://github.com/jbossdemocentral/coolstore-microservice/blob/master/openshift/scripts/README.md is up to date:

--maven-mirror-url  Use the given Maven repository for builds. If not specifid, a Nexus container is deployed in the demo

I.e., I have not set up any other external Maven repo nor provided the Red Hat GA repo URL.

Environment:

$> uname -a
Linux w701-f25 4.10.5-200.fc25.x86_64 #1 SMP Wed Mar 22 20:37:08 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$>oc version
oc v3.4.1.10
kubernetes v1.4.0+776c994
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://127.0.0.1:8443
openshift v3.4.1.10
kubernetes v1.4.0+776c994

Followed these steps using oc cluster:

# cd <coolstore-microservice-clone-root>
oc-cluster up coolstore --image="registry.access.redhat.com/openshift3/ose" --version=v3.4.1.10
# after oc cluster was finally up
oc login -u system:admin
oc project default
oc adm policy add-cluster-role-to-user cluster-admin developer
oc adm policy add-cluster-role-to-user cluster-admin admin

oc login -u developer
# added user param to hopefully avoid any confusion
./openshift/scripts/provision-demo.sh --project-suffix bank --user developer

The nexus pod log only says:

--> Scaling nexus-2 to 1
--> Waiting up to 10m0s for pods in deployment nexus-2 to become ready
error: update acceptor rejected nexus-2: pods for deployment "nexus-2" took longer than 600 seconds to become ready

Any troubleshooting idea here? This demo just doesn't run on my laptop.

Thanks and best whishes,

Karsten

jamesfalkner commented 7 years ago

Did you setup some Persistent Volumes on your OpenShift cluster? If not, you can use the --ephemeral argument to the provisioning script. I bet Nexus is sitting around waiting for its Persistent Volume Claim to attach to a Persistent Volume (which my guess is doesn't exist).

karstengresch commented 7 years ago

Hi James,

thanks for your response, very appreciated!

After...

# removed all previous oc-cluster wrapper configuration - beware!`
rm -Rf ~/.oc
oc-cluster up coolstore --image="registry.access.redhat.com/openshift3/ose" --version=v3.4.1.10
# after oc cluster was finally up
oc login -u system:admin
oc project default
oc adm policy add-cluster-role-to-user cluster-admin developer
oc adm policy add-cluster-role-to-user cluster-admin admin
# after waiting for cluster being ready
oc-cluster create-volume nexus-data 10Gi /home/karsten/.oc/profiles/coolstore/volumes/nexus-data

...I get in the pod log...

--> Scaled older deployment nexus-1 down
--> Scaling nexus-2 to 1
--> Waiting up to 10m0s for pods in deployment nexus-2 to become ready
--> post: Running hook pod ...
/bin/bash: /tmp/nexus-functions: No such file or directory
/bin/bash: add_nexus2_redhat_repos: command not found
--> post: Failed
error: post hook failed: the post hook failed: , aborting deployment: ci-bank/nexus-2

whereas sysout says...

########################################################################
Multi-product MSA Demo (Di 4. Apr 00:57:30 CEST 2017)
########################################################################
Deploying MSA demo...

########################################################################
Creating project...
########################################################################
Creating project ci-bank
Creating project coolstore-test-bank
Creating project coolstore-prod-bank
Creating project inventory-dev-bank
Creating project developer-bank

########################################################################
Configuration
########################################################################
OpenShift master:    https://127.0.0.1:8443
Current user:        developer
Minimal setup:       false
Ephemeral:           false
Project suffix:      bank
GitHub repo:         https://github.com/jbossdemocentral/coolstore-microservice
GitHub branch/tag:   master
Gogs url:            http://gogs-ci-bank.apps.127.0.0.1.nip.io
Gogs admin user:     team
Gogs admin pwd:      team
Gogs user:           developer
Gogs pwd:            developer
Gogs webhook secret: UfW7gQ6Jx4
Maven mirror url:    http://nexus.ci-bank.svc.cluster.local:8081/content/groups/public

########################################################################
Deploying Sonatype Nexus repository manager...
########################################################################
Using template https://raw.githubusercontent.com/OpenShiftDemos/nexus/master/nexus2-persistent-template.yaml
imagestream "nexus" created
deploymentconfig "nexus" created
service "nexus" created
route "nexus" created
persistentvolumeclaim "nexus-pv" created
deploymentconfig "nexus" resource requirements updated
Waiting for Nexus to be ready...
.
(...)
.
Nexus is ready.
Using build template https://raw.githubusercontent.com/jbossdemocentral/coolstore-microservice/master/openshift/templates/coolstore-builds-template.yaml
imagestream "web-ui" created
buildconfig "web-ui" created
imagestream "fis-java-openshift" created
imagestream "coolstore-gw" created
buildconfig "coolstore-gw" created
imagestream "inventory" created
buildconfig "inventory" created
imagestream "catalog" created
buildconfig "catalog" created
imagestream "cart" created
buildconfig "cart" created
imagestream "pricing" created
buildconfig "pricing" created
build "web-ui-1" started
Waiting for web-ui build to be ready...
.
(...)
.
web-ui build still not ready, I GIVE UP!

I.e. the CLI output seems wrong as Nexus never came up.

Nexus pod status is:

$>oc project ci-bank
$>oc status
In project CI/CD (ci-bank) on server https://127.0.0.1:8443

http://nexus-ci-bank.apps.127.0.0.1.nip.io to pod port 8081-tcp (svc/nexus)
  dc/nexus deploys istag/nexus:2.14.2 
    deployment #2 failed 11 minutes ago: config change
    deployment #1 failed 11 minutes ago: newer deployment was found running

Errors:
  * route/nexus is routing traffic to svc/nexus, but either the administrator has not installed a router or the router is not selecting this route.

Does it need to be a shared volume? I'll give it a try and tell later...

Best whishes,

Karsten

jamesfalkner commented 7 years ago

Several services use non-shared PVCs so you need to create about 7-8 PVs (not shared). I'm not sure why it would be unable to run the pod hook if the container started up successfully.

Have you tried --ephemeral ? I know it's not a final solution but would eliminate other issues.

karstengresch commented 7 years ago

Thanks for your immediate response again!

Several services use non-shared PVCs so you need to create about 7-8 PVs (not shared).

Do any naming conventions exist here? If so which - I just wanted to follow the documentation which gave me the impression that running the provisioning script was just everything need. Happy to update the docs so more people could make use of this great demo.

In the meantime: Using shared volumes resulted again in...

Waiting for Nexus to be ready...
.
(...)
.
Nexus still not ready, I GIVE UP!
karstengresch commented 7 years ago

Final comment for today (2 am here, need to get the kids to school today):

Ran... (that is, with --ephemeral)

sudo rm -Rf ~/.oc/
oc-cluster up coolstore --image="registry.access.redhat.com/openshift3/ose" --version=v3.4.1.10
# oc cluster available
oc login -u system:admin
oc project default
oc adm policy add-cluster-role-to-user cluster-admin developer
oc adm policy add-cluster-role-to-user cluster-admin admin
oc login -u developer
./openshift/scripts/provision-demo.sh --project-suffix bank --user developer --ephemeral

with this sysout:

~ ~/dev/rhsa/oscp/coolstore-microservice

########################################################################
Multi-product MSA Demo (Di 4. Apr 01:53:00 CEST 2017)
########################################################################
Deploying MSA demo...

########################################################################
Creating project...
########################################################################
Creating project ci-bank
Creating project coolstore-test-bank
Creating project coolstore-prod-bank
Creating project inventory-dev-bank
Creating project developer-bank

########################################################################
Configuration
########################################################################
OpenShift master:    https://127.0.0.1:8443
Current user:        developer
Minimal setup:       false
Ephemeral:           true
Project suffix:      bank
GitHub repo:         https://github.com/jbossdemocentral/coolstore-microservice
GitHub branch/tag:   master
Gogs url:            http://gogs-ci-bank.apps.127.0.0.1.nip.io
Gogs admin user:     team
Gogs admin pwd:      team
Gogs user:           developer
Gogs pwd:            developer
Gogs webhook secret: UfW7gQ6Jx4
Maven mirror url:    http://nexus.ci-bank.svc.cluster.local:8081/content/groups/public

########################################################################
Deploying Sonatype Nexus repository manager...
########################################################################
Using template https://raw.githubusercontent.com/OpenShiftDemos/nexus/master/nexus2-template.yaml
imagestream "nexus" created
deploymentconfig "nexus" created
service "nexus" created
route "nexus" created
deploymentconfig "nexus" resource requirements updated
Waiting for Nexus to be ready...
.
(...)
.
Nexus is ready.
Using build template https://raw.githubusercontent.com/jbossdemocentral/coolstore-microservice/master/openshift/templates/coolstore-builds-template.yaml
imagestream "web-ui" created
buildconfig "web-ui" created
imagestream "fis-java-openshift" created
imagestream "coolstore-gw" created
buildconfig "coolstore-gw" created
imagestream "inventory" created
buildconfig "inventory" created
imagestream "catalog" created
buildconfig "catalog" created
imagestream "cart" created
buildconfig "cart" created
imagestream "pricing" created
buildconfig "pricing" created
build "web-ui-1" started
Waiting for web-ui build to be ready...
.
(...)
.
web-ui build still not ready, I GIVE UP!

But again, Nexus is not up, same as above (with local persistent volumes).

For the moment, I give up, too, but am very grateful for any troubleshooting advice.

Even better would be an information regarding your cluster setup against the provisioning script is - which I assume - successfully running. Do you use a dedicated environment or does it run on all your laptops, as @siamaksade 's hint in issue https://github.com/jbossdemocentral/coolstore-microservice/issues/45 ,

This demo runs on your laptop :)

suggests??? I really wonder how and think some more information how you bring openshift up could be of high interest for people trying to use the demo.

Best wishes,

Karsten

tqvarnst commented 7 years ago

@karstengresch Can you verify this issue with branch 1.0.x?