search

matthewwall / weewx-influx

uploader for influxdb
GNU General Public License v3.0
45 stars 22 forks source link

create database #19

Closed cstapher closed 5 years ago

cstapher commented 5 years ago

The documentation notes:

When it starts up, this extension will attempt to create the influx database. If credentials for a database administrator were provided, it will use those credentials. Otherwise, it will use the username/password credentials.

But this essentially requires providing admin credentials because non-admin users are not allowed to create databases in influx.

https://docs.influxdata.com/influxdb/v1.7/administration/authentication_and_authorization/#user-types-and-privileges

telegraf, an influx software, includes an option in the 'outputs.influxdb' to skip database creation (skip_database_creation = false). This is provided, presumably for the same reason I've noted with regards to non-admin users.

https://github.com/influxdata/telegraf/tree/master/plugins/outputs/influxdb

Can you provide a similar option to skip_database_creation?

matthewwall commented 5 years ago

added option create_database at commit 3d878b8

cstapher commented 5 years ago

Thank you! This is fantastic. I am encountering an error (see below) but I think I've found the source. I think it's coming from the newline character in the encodestring (Invalid header value 'Basic aW5mbHV4X3dlZXd4OmlhbXdlZXd4\n')

If I comment out line 400 I see "create database failed: HTTP Error 401: Unauthorized" in the weewx logs but I'm still able to successfully write records.

line 400 # req.add_header("Authorization", "Basic %s" % b64s) 


Jan  5 13:46:06 mogwai weewx[2438]: restx: Influx: service version is 0.12
Jan  5 13:46:06 mogwai weewx[2438]: restx: Influx: database is weewxdb
Jan  5 13:46:06 mogwai weewx[2438]: restx: Influx: tags None
Jan  5 13:46:06 mogwai weewx[2438]: restx: Influx: binding is archive
Jan  5 13:46:06 mogwai influxd: [httpd] 127.0.0.1 - - [05/Jan/2019:13:46:06 -0800] "GET /query?q=CREATE+DATABASE+weewxdb HTTP/1.1" 401 55 "-" "weewx/3.7.1" 4e3b4168-1133-11e9-9f3c-6cfdb93bbcf5 80
Jan  5 13:46:06 mogwai weewx[2438]: restx: Influx: create database failed: HTTP Error 401: Unauthorized
Jan  5 13:46:06 mogwai weewx[2438]: restx: Influx: Data will be uploaded to http://localhost:8086

-----
Jan  5 13:50:18 mogwai weewx[2438]: restx: Influx: Published record 2019-01-05 13:50:00 PST (1546725000)

-----

Jan  5 13:41:19 mogwai weewx[26857]: engine: Caught unrecoverable exception in engine:
Jan  5 13:41:19 mogwai weewx[26857]:    ****  Invalid header value 'Basic aW5mbHV4X3dlZXd4OmlhbXdlZXd4\n'
Jan  5 13:41:19 mogwai weewx[26857]:    ****  Traceback (most recent call last):
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/share/weewx/weewx/engine.py", line 865, in main
Jan  5 13:41:19 mogwai weewx[26857]:    ****      engine = engine_class(config_dict)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/share/weewx/weewx/engine.py", line 77, in __init__
Jan  5 13:41:19 mogwai weewx[26857]:    ****      self.loadServices(config_dict)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/share/weewx/weewx/engine.py", line 141, in loadServices
Jan  5 13:41:19 mogwai weewx[26857]:    ****      self.service_obj.append(weeutil.weeutil._get_object(svc)(self, config_dict))
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/share/weewx/user/influx.py", line 309, in __init__
Jan  5 13:41:19 mogwai weewx[26857]:    ****      data_thread = InfluxThread(data_queue, **site_dict)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/share/weewx/user/influx.py", line 387, in __init__
Jan  5 13:41:19 mogwai weewx[26857]:    ****      self.create_database(uname, pword)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/share/weewx/user/influx.py", line 402, in create_database
Jan  5 13:41:19 mogwai weewx[26857]:    ****      self.post_request(req)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/share/weewx/user/influx.py", line 452, in post_request
Jan  5 13:41:19 mogwai weewx[26857]:    ****      return urllib2.urlopen(request, data=payload, timeout=self.timeout)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen
Jan  5 13:41:19 mogwai weewx[26857]:    ****      return opener.open(url, data, timeout)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/lib64/python2.7/urllib2.py", line 431, in open
Jan  5 13:41:19 mogwai weewx[26857]:    ****      response = self._open(req, data)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
Jan  5 13:41:19 mogwai weewx[26857]:    ****      '_open', req)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
Jan  5 13:41:19 mogwai weewx[26857]:    ****      result = func(*args)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/lib64/python2.7/urllib2.py", line 1244, in http_open
Jan  5 13:41:19 mogwai weewx[26857]:    ****      return self.do_open(httplib.HTTPConnection, req)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/lib64/python2.7/urllib2.py", line 1211, in do_open
Jan  5 13:41:19 mogwai weewx[26857]:    ****      h.request(req.get_method(), req.get_selector(), req.data, headers)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/lib64/python2.7/httplib.py", line 1041, in request
Jan  5 13:41:19 mogwai weewx[26857]:    ****      self._send_request(method, url, body, headers)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/lib64/python2.7/httplib.py", line 1074, in _send_request
Jan  5 13:41:19 mogwai weewx[26857]:    ****      self.putheader(hdr, value)
Jan  5 13:41:19 mogwai weewx[26857]:    ****    File "/usr/lib64/python2.7/httplib.py", line 1019, in putheader
Jan  5 13:41:19 mogwai weewx[26857]:    ****      raise ValueError('Invalid header value %r' % (one_value,))
Jan  5 13:41:19 mogwai weewx[26857]:    ****  ValueError: Invalid header value 'Basic aW5mbHV4X3dlZXd4OmlhbXdlZXd4\n'
Jan  5 13:41:19 mogwai weewx[26857]:    ****  Exiting.
cstapher commented 5 years ago

Also, these are the values I'm using in weewx.conf:

    [[Influx]]
        host = localhost
        database = weewxdb
        port = 8086
        username = influx_weewx
        password = foobarwee
        # dbadmin_username = admin
        # dbadmin_password = foobarfoo
        create_database=False
matthewwall commented 5 years ago

oops. i did the newline stripping in the wrong place. do a pull so that you get aec381b then let me know how it goes.

coolbox13 commented 4 years ago

I am encountering a somawhat similar issue. The influxdb is running within a docker container. I can approach my databases without credentials from node-red which is running within an other container. But apparently not from another computer, although i am using admin credentials.

See the log: Mar 9 17:33:26 central weewx[7784]: restx: MQTT: service version is 0.19 Mar 9 17:33:26 central weewx[7784]: restx: MQTT: binding to archive Mar 9 17:33:26 central weewx[7784]: restx: MQTT: topic is weather Mar 9 17:33:26 central weewx[7784]: restx: MQTT: desired unit system is METRIC Mar 9 17:33:26 central weewx[7784]: restx: MQTT: data will be uploaded to mqtt://192.168.178.22:1883/ Mar 9 17:33:26 central weewx[7784]: engine: Finished loading service user.mqtt.MQTT Mar 9 17:33:26 central weewx[7784]: engine: Loading service user.influx.Influx Mar 9 17:33:26 central weewx[7784]: restx: Influx: service version is 0.12 Mar 9 17:33:26 central weewx[7784]: restx: Influx: database is weewx Mar 9 17:33:26 central weewx[7784]: restx: Influx: desired unit system is METRICWX Mar 9 17:33:26 central weewx[7784]: restx: Influx: tags None Mar 9 17:33:26 central weewx[7784]: restx: Influx: binding is archive Mar 9 17:33:26 central weewx[7784]: restx: Influx: create database failed: <urlopen error [Errno 111] Connection refused> Mar 9 17:33:26 central weewx[7784]: restx: Influx: Data will be uploaded to http://192.168.178.32

my setup in weewx.conf is: [[Influx]] server_url = http://192.168.178.32 dbadmin_username = admin dbadmin_password = admin database = weewx measurement = weewx unit_system = METRICWX

In Influxdb already created the weewx database.

any suggestions? thanks in advance