Docker health check failed with internal CA based certificate

SnDsound commented 3 years ago

Hi,

I've 2 instances of graylog, one with version 3.1.4-1 and second with 4.0.5-1. Both are installed as docker container and have the same problem. I'm using HTTPS protocol, with certificate signed with our company CA. Certificates are imported to JKS. Graylog web and api works, but docker shows unhealthy status.

# docker inspect graylog-test
            "Health": {
                "Status": "unhealthy",
                "FailingStreak": 80,
                "Log": [
                    {
                        "Start": "2021-03-26T16:55:57.297102308+01:00",
                        "End": "2021-03-26T16:55:57.598581684+01:00",
                        "ExitCode": 1,
                        "Output": ""
                    },
                    {
                        "Start": "2021-03-26T16:56:07.600596718+01:00",
                        "End": "2021-03-26T16:56:07.916777755+01:00",
                        "ExitCode": 1,
                        "Output": ""
                    },

In my opinion problem is with this case in health_check.sh. Output from this command executed inside container is empty:

graylog@myserver:/$ /health_check.sh
graylog@myserver:/$

Those line are executed (tested by echo inside loops):

if [[ -z "${check_url}" ]]
then
        echo "Not possible to get Graylog listen URI - abort"
        exit 1
fi

if curl --silent --fail "${check_url}"/api
then
        exit 0
fi

my check_url is: https://myserver.domain.my:10000/ If I modified this line:

from
if curl --silent --fail "${check_url}"/api
to:
if curl --fail "${check_url}"/api

I got this error:

curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

But as --silent options is there, error is not present, and container is unhealty. Changing this:

from
if curl --silent --fail "${check_url}"/api
to:
if curl --silent --insecure --fail "${check_url}"/api

and executing this script give me output, and container changed status to healthy.

graylog@myserver:/$ /health_check.sh
https://myserver.domain.my:10000/
<!DOCTYPE html>
<html>
  <head>
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="robots" content="noindex, nofollow">
    <meta charset="UTF-8">
    <title>Graylog Web Interface</title>
    <link rel="shortcut icon" href="https://myserver.domain.my:10000/assets/favicon.png">

  </head>
  <body>
    <script src="https://myserver.domain.my:10000/config.js"></script>

    <script src="https://myserver.domain.my:10000/assets/vendor.594b2a39cb22b445205e.js"></script>

    <script src="https://myserver.domain.my:10000/assets/polyfill.93b51f9ca3be4394b382.js"></script>

    <script src="https://myserver.domain.my:10000/assets/builtins.93b51f9ca3be4394b382.js"></script>

    <script src="https://myserver.domain.my:10000/assets/plugin/org.graylog.plugins.threatintel.ThreatIntelPlugin/plugin.org.graylog.plugins.threatintel.ThreatIntelPlugin.f1e161f5a6628d28e101.js"></script>

    <script src="https://myserver.domain.my:10000/assets/plugin/org.graylog.plugins.collector.CollectorPlugin/plugin.org.graylog.plugins.collector.CollectorPlugin.90c134d3bab79d9b3274.js"></script>

    <script src="https://myserver.domain.my:10000/assets/plugin/org.graylog.aws.AWSPlugin/plugin.org.graylog.aws.AWSPlugin.5c03b3ee3c9ceff2fc8f.js"></script>

    <script src="https://myserver.domain.my:10000/assets/app.93b51f9ca3be4394b382.js"></script>

  </body>
</html>

Regards, Peter

malcyon commented 3 years ago

@SnDsound Thanks for the detailed report. I added the fix you suggested.

SnDsound commented 3 years ago

@juju2112, thanks :)

Graylog2 / graylog-docker

Docker health check failed with internal CA based certificate #156