search

regel / loudml

Loud ML is the first open-source AI solution for ICT and IoT automation
Other
298 stars 93 forks source link

ERROR:root:400: Client sent an HTTP request to an HTTPS server. (influxdb) #358

Open tubemeister opened 4 years ago

tubemeister commented 4 years ago

The loudml docker container seems to connect to influxdb using the settings of the chronograf container, not the ones configured in etc/loudml.yml, and reverting to HTTP in the process.

After getting the error message in the title and double checking that my config files all have HTTPS for my influxdb server, I created a couple of CNAMEs and it turns out loudml is trying to connect (with HTTP) to the domain name configured (with HTTPS) in my chronograf environment.

It's also connecting to the "chronograf" database, which seems to be a default.

Configs: loudml.yml.txt docker-compose.yml.txt

Docker-compose log: docker.log

regel commented 4 years ago

The 'ssl' parameter is explicitly given to influxdb python library during initialization. At first glance, it's hard to see if something is missing. The 'chronograf' database is a secondary database used to insert so-called tagged annotations for abnormal data points, and can be renamed in bucket settings.

from influxdb client library:

    :param ssl: use https instead of http to connect to InfluxDB, defaults to
        False
    :type ssl: bool
tubemeister commented 4 years ago

Err, I can't quite follow you.

So how do I configure loudml to use the correct url and ssl settings if it doesn't seem to use the ones in loudml.yml?

tubemeister commented 4 years ago

Still not getting anywhere with this.

Loudml still tries to talk HTTP to the influxdb server url (with HTTPS!) that is configured in the chronograf docker. The chronograf docker does talk HTTPS to influx, that part works fine. But loudml strips off the HTTPS part for some reason.

tubemeister commented 4 years ago

This is still a thing.

Just updated to latest and tried again. Loudml is still trying to talk HTTP to my influxdb while every link to that influxdb is explicitly configured as HTTPS.

loudml_1 | INFO:root:job[f0c4359e-4efa-4b0b-b574-5beb762eb748] starting, nice=5 loudml_1 | INFO:root:connecting to influxdb on domainname:8086, using database 'chronograf' loudml_1 | ERROR:root:400: Client sent an HTTP request to an HTTPS server. loudml_1 | Traceback (most recent call last): loudml_1 | File "/opt/vendor/lib/python3.5/site-packages/loudml/worker.py", line 53, in run loudml_1 | res = getattr(self, func_name)(*args, **kwargs) loudml_1 | File "/opt/vendor/lib/python3.5/site-packages/loudml/worker.py", line 90, in train loudml_1 | tags={'model': model_name}, loudml_1 | File "/opt/vendor/lib/python3.5/site-packages/loudml/influx.py", line 682, in list_anomalies loudml_1 | result = self.annotationdb.query(query) loudml_1 | File "/opt/vendor/lib/python3.5/site-packages/loudml/influx.py", line 417, in annotationdb loudml_1 | self._annotationdb.create_database(db) loudml_1 | File "/opt/vendor/lib/python3.5/site-packages/influxdb/client.py", line 626, in create_database loudml_1 | method="POST") loudml_1 | File "/opt/vendor/lib/python3.5/site-packages/influxdb/client.py", line 450, in query loudml_1 | expected_response_code=expected_response_code loudml_1 | File "/opt/vendor/lib/python3.5/site-packages/influxdb/client.py", line 302, in request loudml_1 | raise InfluxDBClientError(response.content, response.status_code) loudml_1 | influxdb.exceptions.InfluxDBClientError: 400: Client sent an HTTP request to an HTTPS server. loudml_1 | loudml_1 | 172.19.0.1 - - [2020-09-07 06:51:30] "POST /models/telegraf_disk_mean_used_percent_host_detritus_5m/_train?from=now-30d&to=now&output_bucket=loudml&save_output_data=true&flag_abnormal_data=true HTTP/1.1" 202 153 0.041317 loudml_1 | ERROR:root:job[f0c4359e-4efa-4b0b-b574-5beb762eb748] failed: 400: Client sent an HTTP request to an HTTPS server. loudml_1 |

tubemeister commented 4 years ago

Looked into this a bit deeper.

In 'loudml/influx.py', a cfg var shows up in this line:

def __init__(self, cfg):

It seems to contain some influxdb connection info that it's getting from chronograf when you click the "Create Baseline" button, but fields are mismatched or incomplete.

It contains a 'username' field but the influx.py code seems to expect 'dbuser'. It does not contain a password at all, nor is the 'use_ssl' field set which defaults to 'False'.

If I edit influx.py and hardcode the above cfg fields with some appropriate config values it all starts working, but that clearly isn't how this is meant to work.

I'm running this in docker, with the latest images:

tubemeister commented 3 years ago

Hey, is anything still happening with loudml or is this project dead?