Logging working and enabled by default?

lukas-vlcek commented 8 years ago

First, based on the comment in addons-setup script it seems that there should be expected issues with logging functionality. Is this still the case or the logging is working correctly now? If it is not working correctly, what are the issues?

Second, is logging addon expected to be installed by default and running after initial vagrant up? I can see it is running for me but after seeing https://github.com/openshift-evangelists/vagrant-origin/issues/28 I wonder if that was the original plan. However, the default configuration seems to include logging now.

jfmatth commented 8 years ago

I just spun up the latest version and it's close to working, but the route isn't working after I accept the self-signed cert. Then, looking at the archive yields a redirect error :(

lukas-vlcek commented 8 years ago

May be this is the same redirecting issue @richm has been hitting for some time now? Rich do you know if there is opened issue for this somewhere (may be in aggregated logging)?

richm commented 8 years ago

@lukas-vlcek you mean for the kibana issue? I don't know. It's probably a configuration issue - i.e. you have to get the vm and the host machine to have the same forward and reverse resolution for the kibana hostname and IP address. That seems to be tricky to get right, and any guidance from anyone who has been able to do this would be very appreciated.

jorgemoralespou commented 8 years ago

@lukas-vlcek The comment is there, since I still need to figure out what's the best way to install logging. It seems to change with every release, but latests should be working (Still need to try this again). The issue #28 adds scripts to enable/disable logging. What these do is scaling them to 0 or back to 1, but the pods need to be installed at least once with the initial setup, which is what happens.

So, in summary, default install will have logging enabled, but can be disabled (although still installed). You can of course, choose not to install.

I'll retry the install and if everything works will close this issue.

jfmatth commented 8 years ago

Just spun up a new instance, and the LOGS tab is gone entirely :(

What happened?

However, when i start a project, i can view logs, but the Archive link gives me an auth error

require "redirect" parameter beginning with "/"

iconoeugen commented 8 years ago

@jfmatth and @lukas-vlcek: after PR merge that provided the fix for #39, the logs and even the "View archive" works for me, but only with Openshift release v1.2.1. With latest relase or master I have the same behaviour.

@jorgemoralespou This seems to point to a problem in open shift and not necessarily on how logging is configured with these scripts.

iconoeugen commented 8 years ago

With the default configuration:

export __OS_ORIGIN_BRANCH="master"
export __OS_IMAGES_VERSION=latest

the deployment is compiling the last comit, but is downloading the images tagged with v1.2.1

I got it working either by compiling the images export __OS_BUILD_IMAGES=true or using the configuration:

export __OS_ORIGIN_BRANCH="master"
export __OS_IMAGES_VERSION="v1.3.0-alpha.2"

jorgemoralespou commented 8 years ago

That's a problem as latest are just tagged after 1.2.1 got released. I hope this will get fixed soon. Will open an issue in origin.

El 21 jul. 2016 22:08, "Horatiu Eugen Vlad" notifications@github.com escribió:

With the default configuration:

export OS_ORIGIN_BRANCH="master" export OS_IMAGES_VERSION=latest

the deployment is compiling the last comit, but is downloading the images tagged with v1.2.1

I got it working either by compiling the images or using the configuration:

export OS_ORIGIN_BRANCH="master" export OS_IMAGES_VERSION="v1.3.0-alpha.2"

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/openshift-evangelists/vagrant-origin/issues/33#issuecomment-234368414, or mute the thread https://github.com/notifications/unsubscribe-auth/AAEyDn2S7MtHyshC8UevST9lSah4_shaks5qX9HAgaJpZM4JMj16 .

jorgemoralespou commented 8 years ago

Just seen again and latest should not be the same as 1.2.1, as a latest is being built and push every day.

Maybe an error in the script. Will check tomorrow.

jfmatth commented 8 years ago

Just checked the latest version, and after a bit of time (on my 8gb Windows laptop) Kibana came up, lovely.

lukas-vlcek commented 8 years ago

I am still not able to access Kibana UI. Probably I am doing something wrong. I haven't installed any app so far, I am just talking about OOB state after vagrant up and after oc get pods indicate all deployments are finished:

# oc whoami
system:admin

# oc project
Using project "openshift-infra" on server "https://10.2.2.2:8443".

# oc get pods
NAME                          READY     STATUS      RESTARTS   AGE
hawkular-cassandra-1-jst6i    1/1       Running     0          10m
hawkular-metrics-9upxv        1/1       Running     0          10m
heapster-thfmm                1/1       Running     1          10m
logging-curator-1-tx6wp       1/1       Running     0          10m
logging-deployer-rr15x        0/1       Completed   0          12m
logging-es-jg81kanm-1-lqxqs   1/1       Running     0          10m
logging-fluentd-9ocx0         1/1       Running     0          10m
logging-kibana-1-tnisp        2/2       Running     0          10m
metrics-deployer-aag8p        0/1       Completed   0          12m

The way I understand logging deployment it is the following:

There is openshift-infra project, that hosts single EFK stack but there are defined two kibana hosts and two logging services groups:
- one for apps developers
- second for OPS admins (anything matching *_ops*)

# oc get route
NAME                 HOST/PORT                               PATH      SERVICE                           TERMINATION   LABELS
hawkular-metrics     hawkular-metrics.apps.10.2.2.2.xip.io             hawkular-metrics:https-endpoint   reencrypt     metrics-infra=hawkular-metrics,name=hawkular-metrics
logging-kibana       kibana.apps.10.2.2.2.xip.io                       logging-kibana                    reencrypt     component=support,logging-infra=support,provider=openshift
logging-kibana-ops   kibana-ops.apps.10.2.2.2.xip.io                   logging-kibana-ops                reencrypt     component=support,logging-infra=support,provider=openshift

# oc get services
NAME                       CLUSTER-IP       EXTERNAL-IP   PORT(S)                               AGE
hawkular-cassandra         172.30.1.13      <none>        9042/TCP,9160/TCP,7000/TCP,7001/TCP   13m
hawkular-cassandra-nodes   None             <none>        9042/TCP,9160/TCP,7000/TCP,7001/TCP   13m
hawkular-metrics           172.30.37.186    <none>        443/TCP                               13m
heapster                   172.30.142.133   <none>        80/TCP                                12m
logging-es                 172.30.180.197   <none>        9200/TCP                              13m
logging-es-cluster         None             <none>        9300/TCP                              13m
logging-es-ops             172.30.162.176   <none>        9200/TCP                              13m
logging-es-ops-cluster     None             <none>        9300/TCP                              13m
logging-kibana             172.30.45.63     <none>        443/TCP                               13m
logging-kibana-ops         172.30.88.79     <none>        443/TCP                               13m

Now, in openshift console it seems that logging-kibana should be accessible at https://kibana.apps.10.2.2.2.xip.io/:

But accessing https://kibana.apps.10.2.2.2.xip.io/ in browser gives me error:

This site can’t be reached. kibana.apps.10.2.2.2.xip.io refused to connect.

My configuration of hosts:

$ cat /private/etc/hosts # (hint: I am on Mac)
127.0.0.1 localhost kibana.apps.10.2.2.2.xip.io 
255.255.255.255 broadcasthost
::1             localhost

When I try to connect to kibana from within the VM machine (as root) it returns 302 - Moved Temporarily:

[root@origin ~]# curl -v -s -k https://kibana.apps.10.2.2.2.xip.io
* About to connect() to kibana.apps.10.2.2.2.xip.io port 443 (#0)
*   Trying 10.2.2.2...
* Connected to kibana.apps.10.2.2.2.xip.io (10.2.2.2) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*   subject: E=root@ip-10-35-63-192,CN=ip-10-35-63-192,OU=SomeOrganizationalUnit,O=SomeOrganization,L=SomeCity,ST=SomeState,C=--
*   start date: Feb 24 05:56:46 2014 GMT
*   expire date: Feb 24 05:56:46 2015 GMT
*   common name: ip-10-35-63-192
*   issuer: E=root@ip-10-35-63-192,CN=ip-10-35-63-192,OU=SomeOrganizationalUnit,O=SomeOrganization,L=SomeCity,ST=SomeState,C=--
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: kibana.apps.10.2.2.2.xip.io
> Accept: */*
> 
< HTTP/1.1 302 Moved Temporarily
< X-Powered-By: Express
< Location: https://10.2.2.2:8443/oauth/authorize?response_type=code&redirect_uri=https%3A%2F%2Fkibana.apps.10.2.2.2.xip.io%2Fauth%2Fopenshift%2Fcallback&client_id=kibana-proxy
< Content-Length: 0
< Set-Cookie: openshift-auth-proxy-session=j2r-HqX0EOQyPfJG4yqvmw.mSnU_HO3kdqaA2txjFXaHPh6YcJ2L3xhPkSQmrLzcFVpIP3wLdXohJfLKQKLsqcMmD9MzwmdNN36wxZusI4gMg.1469192238549.3600000.dzaAvaP3oEEtMT0fZVcsqUfAepyiK07nw2GIFXSUDRs; path=/; secure; httponly
< Date: Fri, 22 Jul 2016 12:57:18 GMT
< Set-Cookie: 884b5a94f1070fac466538caeec4cf7b=f052203ceec41d220adef761b846a33c; path=/; HttpOnly; Secure
< 
* Connection #0 to host kibana.apps.10.2.2.2.xip.io left intact

I am kinda lost at this point, does it mean I need to provide security certs to curl command and to web browser?

richm commented 8 years ago

@lukas-vlcek curl does not follow redirects (30x return with Location:) automatically. You'll have to use a browser or some sort of redirect-aware client.

jorgemoralespou commented 8 years ago

curl -L does

iconoeugen commented 8 years ago

@lukas-vlcek One config issue that I can see is in your /etc/hosts, try using the following entry:

10.2.2.2 kibana.apps.10.2.2.2.xip.io

Openshift is running inside the guest VM, so when you are on your host machine you must access the guest VM IP address and not the localhost of the host

jfmatth commented 8 years ago

I've tried the latest build and although it took a while, it does finally come up.

Can you get to the cockpit app? http://cockpit.apps.10.2.2.2.xip.io/

I found that if cockpit works, then routing is working and you shouldn't have to change anything in the hosts file. Also, mine only works with Chrome for some reason, not Edge or Firefox. I'm a Windows user.

I did have to wait a while after everything was built for Kibana to come up. I gave it 10min after the build, so it could download everything.

Did you check the events for the Openshift-infra project?

openshift-evangelists / vagrant-origin

Logging working and enabled by default? #33