search

influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.96k stars 3.56k forks source link

Does docker InfluxDB v1.8 support collectD with security-level = "encrypt" ? #19377

Open systemcrash opened 4 years ago

systemcrash commented 4 years ago

Nothing I see gives me confidence that it does. I read some old reports here. PR added functionality from v1.2 - Check.

I run docker pull influxdb for the :latest tag, and it only pulls in 1.8.1 when release 1.8.2 says it should be latest...

Docs: https://docs.influxdata.com/influxdb/v1.8/supported_protocols/collectd/

security-level = "none" # "none", "sign", or "encrypt"

BUT

https://docs.influxdata.com/influxdb/v1.8/administration/config/#collectd-settings says: security-level = "none"

but https://docs.influxdata.com/enterprise_influxdb/v1.8/administration/config-data-nodes#collectd-settings says:

security-level = ""

The collectd security level can be “” (or “none”), “sign”, or “encrypt”.

Enterprise only feature????

I run docker with my env.file so:

INFLUXDB_DATA_ENGINE=tsm1
INFLUXDB_REPORTING_DISABLED=false
INFLUXDB_LOGGING_LEVEL=debug
INFLUXDB_COLLECTD_ENABLED=true
INFLUXDB_COLLECTD_DATABASE=collectd
INFLUXDB_COLLECTD_BIND_ADDRESS=:25826
INFLUXDB_COLLECTD_SECURITY_LEVEL=encrypt
INFLUXDB_COLLECTD_AUTH_FILE=/etc/collectd/auth_file
INFLUXDB_COLLECTD_TYPESDB=/usr/share/collectd/types.db

and my auth_file appropriately set for:

user: password

but when I run docker-compose exec influxdb sh and look around, the exports are there:

# set
HOME='/root'
HOSTNAME='influxdb'
IFS='
'
INFLUXDB_COLLECTD_AUTH_FILE='/etc/collectd/auth_file'
INFLUXDB_COLLECTD_BIND_ADDRESS=':25826'
INFLUXDB_COLLECTD_DATABASE='collectd'
INFLUXDB_COLLECTD_ENABLED='true'
INFLUXDB_COLLECTD_SECURITY_LEVEL='encrypt'
INFLUXDB_COLLECTD_TYPESDB='/usr/share/collectd/types.db'
INFLUXDB_DATA_ENGINE='tsm1'
INFLUXDB_LOGGING_LEVEL='debug'
INFLUXDB_REPORTING_DISABLED='false'
INFLUXDB_VERSION='1.8.1'
OPTIND='1'
PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
PPID='0'
PS1='# '
PS2='> '
PS4='+ '
PWD='/'
TERM='xterm'

but the config doesn't seem to reflect the settings:

# cat /etc/influxdb/influxdb.conf
[meta]
  dir = "/var/lib/influxdb/meta"

[data]
  dir = "/var/lib/influxdb/data"
  engine = "tsm1"
  wal-dir = "/var/lib/influxdb/wal"

If I set INFLUXDB_COLLECTD_SECURITY_LEVEL=Encrypt the docker image wont start, and reports the error

run: invalid collectd config: Invalid security level. To generate a valid configuration file run `influxd config > influxdb.generated.conf`

But if I set INFLUXDB_COLLECTD_SECURITY_LEVEL=encrypt it starts, but no indication or log messages that it takes this:

ts=2020-08-19T13:58:13.756871Z lvl=info msg="InfluxDB starting" log_id=0Oil2w~0000 version=1.8.1 branch=1.8 commit=af0237819ab9c5997c1c0144862dc762b9d8fc25
ts=2020-08-19T13:58:13.756957Z lvl=info msg="Go runtime" log_id=0Oil2w~0000 version=go1.13.8 maxprocs=4
ts=2020-08-19T13:58:13.858966Z lvl=info msg="Using data dir" log_id=0Oil2w~0000 service=store path=/var/lib/influxdb/data
ts=2020-08-19T13:58:13.859028Z lvl=info msg="Compaction settings" log_id=0Oil2w~0000 service=store max_concurrent_compactions=2 throughput_bytes_per_second=50331648 throughput_bytes_per_second_burst=50331648
ts=2020-08-19T13:58:13.859051Z lvl=info msg="Open store (start)" log_id=0Oil2w~0000 service=store trace_id=0Oil2xOl000 op_name=tsdb_open op_event=start
ts=2020-08-19T13:58:13.888044Z lvl=info msg="Reading file" log_id=0Oil2w~0000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/2/_00001.wal size=10486175
ts=2020-08-19T13:58:13.888313Z lvl=info msg="Opened file" log_id=0Oil2w~0000 engine=tsm1 service=filestore path=/var/lib/influxdb/data/_internal/monitor/1/000000001-000000001.tsm id=0 duration=0.187ms
ts=2020-08-19T13:58:13.967251Z lvl=info msg="Opened shard" log_id=0Oil2w~0000 service=store trace_id=0Oil2xOl000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/_internal/monitor/1 duration=80.203ms
ts=2020-08-19T13:58:14.734679Z lvl=info msg="Reading file" log_id=0Oil2w~0000 engine=tsm1 service=cacheloader path=/var/lib/influxdb/wal/_internal/monitor/2/_00002.wal size=8533262
ts=2020-08-19T13:58:15.503957Z lvl=info msg="Opened shard" log_id=0Oil2w~0000 service=store trace_id=0Oil2xOl000 op_name=tsdb_open index_version=inmem path=/var/lib/influxdb/data/_internal/monitor/2 duration=1616.972ms
ts=2020-08-19T13:58:15.504319Z lvl=info msg="Open store (end)" log_id=0Oil2w~0000 service=store trace_id=0Oil2xOl000 op_name=tsdb_open op_event=end op_elapsed=1645.265ms
ts=2020-08-19T13:58:15.504401Z lvl=info msg="Opened service" log_id=0Oil2w~0000 service=subscriber
ts=2020-08-19T13:58:15.504420Z lvl=info msg="Starting monitor service" log_id=0Oil2w~0000 service=monitor
ts=2020-08-19T13:58:15.504432Z lvl=info msg="Registered diagnostics client" log_id=0Oil2w~0000 service=monitor name=build
ts=2020-08-19T13:58:15.504441Z lvl=info msg="Registered diagnostics client" log_id=0Oil2w~0000 service=monitor name=runtime
ts=2020-08-19T13:58:15.504455Z lvl=info msg="Registered diagnostics client" log_id=0Oil2w~0000 service=monitor name=network
ts=2020-08-19T13:58:15.504464Z lvl=info msg="Registered diagnostics client" log_id=0Oil2w~0000 service=monitor name=system
ts=2020-08-19T13:58:15.504503Z lvl=info msg="Starting precreation service" log_id=0Oil2w~0000 service=shard-precreation check_interval=10m advance_period=30m
ts=2020-08-19T13:58:15.504522Z lvl=info msg="Starting snapshot service" log_id=0Oil2w~0000 service=snapshot
ts=2020-08-19T13:58:15.504533Z lvl=info msg="Starting continuous query service" log_id=0Oil2w~0000 service=continuous_querier
ts=2020-08-19T13:58:15.504545Z lvl=info msg="Starting HTTP service" log_id=0Oil2w~0000 service=httpd authentication=false
ts=2020-08-19T13:58:15.504555Z lvl=info msg="opened HTTP access log" log_id=0Oil2w~0000 service=httpd path=stderr
ts=2020-08-19T13:58:15.504564Z lvl=info msg="Storing statistics" log_id=0Oil2w~0000 service=monitor db_instance=_internal db_rp=monitor interval=10s
ts=2020-08-19T13:58:15.504748Z lvl=info msg="Listening on HTTP" log_id=0Oil2w~0000 service=httpd addr=[::]:8086 https=false
ts=2020-08-19T13:58:15.504802Z lvl=info msg="Starting retention policy enforcement service" log_id=0Oil2w~0000 service=retention check_interval=30m
ts=2020-08-19T13:58:15.504860Z lvl=info msg="Starting collectd service" log_id=0Oil2w~0000 service=collectd
ts=2020-08-19T13:58:15.505105Z lvl=info msg="Loading types from file" log_id=0Oil2w~0000 service=collectd path=/usr/share/collectd/types.db
ts=2020-08-19T13:58:15.506811Z lvl=info msg="Listening on UDP" log_id=0Oil2w~0000 service=collectd addr=[::]:25826
ts=2020-08-19T13:58:15.507277Z lvl=info msg="Listening for signals" log_id=0Oil2w~0000
ts=2020-08-19T13:58:15.508046Z lvl=info msg="Sending usage statistics to usage.influxdata.com" log_id=0Oil2w~0000

The docker image file init-influxdb.sh gives me no confidence that the settings are ever read.

Does the influxd or some other Daemon read the environment variables directly?

output from :8086/debug/vars :

...
"collectd::25826": {"name":"collectd","tags":{"bind":":25826"},"values":{"batchesTx":0,"batchesTxFail":0,"bytesRx":0,"droppedPointsInvalid":0,"pointsParseFail":0,"pointsRx":0,"pointsTx":0,"readFail":0}}
...

tcpdump shows encrypted packets ingressing to my docker image - but the DB remains empty.

I even tried setting INFLUXDB_LOGGING_LEVEL=debug but it does nothing evident in the docker image.

It seems to me that the docker image is neutered and can't do anything with most of the environment variables documented.

Steps to reproduce: List the minimal actions needed to reproduce the behavior.

  1. Configured two docker images, both with collectd. One with encrypt. One without.
  2. Docker without encrypt: db fills. Docker with encrypt. db stays empty.
  3. help?

Expected behavior: influxdb to adhere to the passed environment variables documented everywhere in the docs.

Actual behavior: Nada.

Environment info: Docker - latest.

Config: Copy any non-default config values here or attach the full config as a gist or file.

Logs: No errors. Image does not observe the debug logging environment flag passed.

Performance: Pointless.

systemcrash commented 4 years ago

Docker image currently contains no: /docker-entrypoint-initdb.d/ folder which init scripts indicate otherwise.