Basic Auth Not Working

[steaksauce-]

I have a brand new instance of Status Engine, with the UI, and basic auth is not working. CrateDB is the backend. I show the test users (with simple passwords: 123) in both the Crate database and the /usr/share/statusengine-ui/bin/Console.php users list command. I can't find anything in any sort of log anywhere, except the httpd logs which simply show the request for the login page.

The guide or CentOS7 seems to have been hastily modified from one of the Ubuntu guides, so I'm afraid I may have missed something using the installation guide @ https://statusengine.org/ui/#overview

Below is my config file (comments removed):

[root@localhost ~]# cat /usr/share/statusengine-ui/etc/config.yml | egrep -v "^\s*(#|$)"
allow_anonymous: 0
anonymous_can_submit_commands: 1
urls_without_login:
  - login #looks like /login in the browser's address bar
  - loginstate
auth_type: basic
use_crate: 1
crate:
  nodes:
    - 192.168.111.145:4200
display_perfdata: 0
perfdata_backend: crate
graphite_prefix: statusengine
graphite_url: http://localhost:8080
graphite_use_basic_auth: 0
graphite_user: graphite
graphite_password: password
graphite_allow_self_signed_certificates: 0
graphite_illegal_characters: /[^a-zA-Z^0-9\-\.]/
elasticsearch_index: statusengine-metric-
elasticsearch_pattern: daily
elasticsearch_address: 127.0.0.1
elasticsearch_port: 9200

The kicker is that even when not logged in, I can navigate to the pages not specified in urls_without_login, but I can't view any data. If I enable anonymous login, I still can't view any data. I have yet to check Crate to see if the Naemon data is there though.

Any ideas on getting this auth issue resolved? We are looking at moving away from Thruk and would like to get this rolling.

[nook24]

The guide or CentOS7 seems to have been hastily modified from one of the Ubuntu guides, so I'm afraid I may have missed something using the installation guide @ https://statusengine.org/ui/#overview

Most of the documentation is static. Only relevant part like yum/apt commands are changing when selecting a different OS. I'm not a centos user at all. If you have any suggestions to improve the documentation i would appreciate a pull request: https://github.com/statusengine/docs :)

The kicker is that even when not logged in, I can navigate to the pages not specified in urls_without_login

What do you mean by this exactly? Can you please provide an Example with URL for me so I can verify this on a test system?

Statusengine only protects the API endpoint with an authentication:

Static files like HTML templates and images are not protected:

But I'm not sure if this is the issue you have.

If I enable anonymous login, I still can't view any data. I have yet to check Crate to see if the Naemon data is there though.

Yes, please check your database for records.

In a quick test my Statusengine Ui runs into the following issue (when I use CrateDB as DB Backend):

  "error": "No more servers available, exception from last server: cURL error 7: Failed to connect to 142.93.99.193: Permission denied (see http://curl.haxx.se/libcurl/c/libcurl-errors.html)"

Interesting fact, because of my Statusengine Worker has no connection problems at all.

I will setup a test system this evening (or over the weekend) for deeper investigations.

[steaksauce-]

I'm not a centos user at all. If you have any suggestions to improve the documentation i would appreciate a pull request: https://github.com/statusengine/docs :)

Doc updates are my most frequent contributions (as I'm not actually a dev). Once I hash this out into something repeatable, I will most likely make a PR.

What do you mean by this exactly? Can you please provide an Example with URL for me so I can verify this on a test system?

I will provide a URL late this evening or sometime tomorrow. I can navigate to like the "monitored nodes" or "services" page with basic auth enabled and anonymous login turned off.

But I'm not sure if this is the issue you have.

If I enable anonymous login, I still can't view any data. I have yet to check Crate to see if the Naemon data is there though.

Yes, please check your database for records.

The data is there. I've verified that the workers are working normally.

In a quick test my Statusengine Ui runs into the following issue (when I use CrateDB as DB Backend): "error": "No more servers available, exception from last server: cURL error 7: Failed to connect

I do not see this error. But the symptoms of my problem lead me to believe that either my config file is not being read, or that the UI is not actually connected to the CrateDB. Are there any logs or troubleshooting I can do to confirm either of these?

[nook24]

Could you please verify if the issue als exists on this system: https://165.22.28.247 Username: admin Password: 123

The system is hosted on DigitalOcean so i will delete it soon.

[steaksauce-]

I am able to log onto this system.

Also, <3 DO

[nook24]

I am able to log onto this system.

Perfect. Does this system also have the issue that no login is required? Or other leaking of data?

---

In my case, the Failed to connect to 142.93.99.193: Permission denied error was caused by SELinux.

So I disabled SELinux: https://linuxize.com/post/how-to-disable-selinux-on-centos-7/#check-the-selinux-status

My systems had the following symptoms: I was able to query the users from CrateDB like so:

[root@centos-s-2vcpu-4gb-fra1-01 ~]# /usr/share/statusengine-ui/bin/Console.php users list
+----------+-------------------------------------+
| Username | Password                            |
+----------+-------------------------------------+
| admin    | $2ydkfgjmsdkfm                      |
+----------+-------------------------------------+

But I got the error message above if i tried to load Statusengine Interface through Apache or Nginx/php-fpm.

Statusengine Worker was able to insert check result data into the CrateDB database as well.

After I disable SELinux and reboot the system, everything was working fine.:

My config files:

[root@centos-s-2vcpu-4gb-fra1-01 ~]# cat /etc/sysconfig/crate
# Recommended memory settings for production:
# - assign half of the OS memory to CrateDB
#   (e.g. 26g, stay below ~30G to benefit from CompressedOops)
# - disable swapping my setting bootstrap.mlockall in crate.yml

CRATE_HEAP_SIZE=512m

CRATE_HOME=/opt/crate
CRATE_PATH_LOG=/var/log/crate
CRATE_GC_LOG_DIR=/var/log/crate
CRATE_HEAP_DUMP_PATH=/var/lib/crate
CRATE_PATH_CONF=/etc/crate
CRATE_CONFIG=/etc/crate/crate.yml

CRATE_USE_IPV4=true #<--- Add this

CRATE_INCLUDE=/opt/crate/bin/crate.in.sh

[root@centos-s-2vcpu-4gb-fra1-01 ~]# cat /etc/crate/crate.yml | egrep -v "^\s*(#|$)"
auth.host_based.enabled: true
auth:
  host_based:
    config:
      0:
        user: crate
        address: _local_
        method: trust
      99:
        method: password

node.name: "centos"
license.enterprise: false
network.bind_host: 165.22.28.247
network.publish_host: 165.22.28.247

Statusengine Worker:

[root@centos-s-2vcpu-4gb-fra1-01 etc]# diff config.yml.example config.yml
10c10
< node_name: Crowbar
---
> node_name: SE-Centos
98c98
<     - 127.0.0.1:4200
---
>     - 165.22.28.247:4200
111c111
< max_bulk_delay: 5
---
> max_bulk_delay: 1
149c149
< process_perfdata: 0
---
> process_perfdata: 1

Statusengine Ui:

104c104
<     - 172.0.0.1:4200
---
>     - 165.22.28.247:4200
129c129
< display_perfdata: 0
---
> display_perfdata: 1

Are there any logs or troubleshooting I can do to confirm either of these?

The example Apache config of Statusengine will log any errors to /var/log/apache2/statusengine-ui-error.log. Unfortunately the log was empty in my case :(

Probably an SELinux issue on your system too

[steaksauce-]

Thanks for the research!

I will take a look into the SELinux being the possibility, but will not be able to do so until tonight or tomorrow afternoon.

I copied the VM from a Thruk test box and I know that SELinux has to be disabled for Thruk, so it may be disabled already.

I will update you with anything I find regarding this.

Thanks again!

[steaksauce-]

I had someone who currently has access to this system run sestatus and getenforce and both show disabled.

I missed this last time:

Perfect. Does this system also have the issue that no login is required? Or other leaking of data?

I don't see any data at all, whether or not anonymous login is enabled or not. Because there's data within the database AND logins are not working, I believe that the UI isn't talking to the Crate at all (or may not be reading the config at all).

Crate IS set to use IPv4. ss and netstat confirms that Crate is listening on IPv4 and not IPv6 (I missed the configuration initially, using IPv6 instead).

My display_perfdata: 0 in /usr/share/statusengine-ui/etc/config.yml is set to 0. Is this the setting required to show monitoring data in the UI? I guess it would make sense, assuming perfdata = monitoring data in this context.

[nook24]

My display_perfdata: 0 in /usr/share/statusengine-ui/etc/config.yml is set to 0. Is this the setting required to show monitoring data in the UI?

This option determines if the interface render Performance Graphs (Charts) or not. So this should not be related to the issue.

Are you using Apache or Nginx? And could you please post your web servers config?

Does /usr/share/statusengine-ui/bin/Console.php users list report the users on your system?

[steaksauce-]

Apache (httpd on CentOS).

My config is a copy of the config from https://statusengine.org/ui/#apache-example-config (Centos 7.5 example). The only things that are changes are the VHosts, the SSL keys, and the ServerName (changed to IP address to keep from having to have custom hosts file).

<VirtualHost 192.168.111.145:80>
    ServerName 192.168.111.145

    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule (.*) https://%{SERVER_NAME}/$1 [R,L]

    DocumentRoot "/usr/share/statusengine-ui/public/"

    RedirectMatch 404 /\.git

    ErrorLog "/var/log/httpd/statusengine-ui-error.log"
    CustomLog "/var/log/httpd/statusengine-ui-access.log" combined
</VirtualHost>

<VirtualHost 192.168.111.145:443>
    ServerName 192.168.111.145

    RedirectMatch 404 /\.git

    DocumentRoot "/usr/share/statusengine-ui/public/"

    SSLEngine On
    #SSL keys specified in conf.d/ssl.conf

    ErrorLog "/var/log/httpd/statusengine-ui-error.log"
    CustomLog "/var/log/httpd/statusengine-ui-access.log" combined
</VirtualHost>

I can view users in both the CrateDB using crash and using the /usr/share/statusengine-ui/bin/Console.php users list command.

[nook24]

I can view users in both the CrateDB using crash and using the /usr/share/statusengine-ui/bin/Console.php users list command.

Perfect! This means that Statusengine UI uses your config file and it is working! At least if executed via the CLI. I bet there is some weird CentOS thing going on. Your Apache config looks good for me. I guess you could remove ServerName if its not an FQDN.

So I guess someting is blocking the connection to CrateDB PHP gets executed though Apache.

---

Just as an experiment: Maybe you should install Statusengine UI on an Ubuntu Bionic system and see if it can connect to your CentOS CrateDB server? Just for testing purpose...

[steaksauce-]

Just as an experiment: Maybe you should install Statusengine UI on an Ubuntu Bionic system and see if it can connect to your CentOS CrateDB server? Just for testing purpose...

I'm pretty new to to Crate. Can you point me in the direction of some documentation to allow remote connections to Crate? My understanding is that Crate does not allow remote connections by default.

As a side note to this experiment, I have firewalld disabled and stopped, iptables is not installed, and ss/netstat show connections from the host to itself on port:4200 (Crate). I guess I could also move the apache config to another directory so it's not read, and see if the number of connections to Crate drops.

EDIT: I guess the connection test route still doesn't show if it's CentOS related, and it seems like the UI is connecting to the DB anyways since users can be added.

EDIT 2: Seems like https://crate.io/docs/crate/reference/en/latest/admin/auth/hba.html (host based auth) might be the solution to allow remote connections to Crate.

[steaksauce-]

I haven't been able to try Ubuntu yet, but a thought comes to mind: when I pulled the UI from github, I think I pulled latest. Which tag is the current stable?

[steaksauce-]

So Ubuntu works, which leads me to believe it's CentOS related. To verify, I left Naemon, Crate, Gearmand, and SE Worker on CentOS. I installed SE-UI on Ubuntu and connected it to Crate on the CentOS box.

I tried to reinstall SE-UI on the Cent box with no luck. I tried a different version of SE-UI with no luck as well. Any ideas?

[nook24]

I'm pretty new to to Crate. Can you point me in the direction of some documentation to allow remote connections to Crate? My understanding is that Crate does not allow remote connections by default.

The relevant config options are:

network.bind_host: 165.22.28.247
network.publish_host: 165.22.28.247

WARNING If you are using Crate Community Edition there is not authentication method. The database can be accessed by everyone. You should definitely create a second private network for the database cluster!

AFAIK host based auth is an Enterprise Feature. On my Crate setup I set license.enterprise: false to disable all enterprise features.

Do you plan to setup a multi node crate cluster? The system is not designed to run as a single node application.

If you want a simple and single node solution, you should my be go with MySQL/MariaDB.

EDIT: I guess the connection test route still doesn't show if it's CentOS related, and it seems like the UI is connecting to the DB anyways since users can be added.

Users got created via the Statusengine command line interface which most likely got executed by root and should be therefore not effected by whatever is causing the issue. At least that's my theory. The UI itself on the other hand gets executed through apache and as apache user.

I pulled the UI from github, I think I pulled latest. Which tag is the current stable?

The master branch is also the current stable version.

I tried to reinstall SE-UI on the Cent box with no luck. I tried a different version of SE-UI with no luck as well. Any ideas?

Unfortunately no. It worked on my CentOS7.6 test bench after I disabled SELinux. I used an DigitalOcean machine so I don't know what defaults where configured and installed by them. I also posted all my config files in https://github.com/statusengine/interface/issues/23#issuecomment-502144049 hoping that could help you tracking down the issue.

Have you already tried to use the php built-in web server to serve Statusengine Ui running as root?

[steaksauce-]

Alright, now I feel like I have wasted a lot of time.

I tracked down the problem.

The issue was drum roll

I had php-cli, php-mysql, and php-ldap installed. However, I did not have normal php installed. 🤦‍♂

I discovered this when doing a sanity check of the package versions installed.