search

influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.95k stars 3.56k forks source link

backup not working with selfsigned certificate #19989

Closed mnsmithuk closed 4 years ago

mnsmithuk commented 4 years ago

I was told today that backup is now working in v2 GA so I tried the influxd backup command today on v2.0.0 and also v2.01.

When https is not enabled I get the expected results when running influxd backup however when a selfsgned certificate is used even with the --skip-verify argument the backup fails with errors pointing to the certificate.

Please note, I am able to sign into the web gui using https with the self-signed certificate.

Steps to reproduce:

influx backup --org-id 2b0bec00ff5d5fab --token Zz_Plltgu0MwAvTI8vnTkADW3O9wk7Zk5OS3w5QDof_5NkshCQfK82z3ezudTCSM55Bi9KnnJMKMZ3r2EKmG4Q== --bucket telegraf --skip-verify /backups/influxdb2_backup

Expected behavior:

influx backup --org-id 2b0bec00ff5d5fab --token Zz_Plltgu0MwAvTI8vnTkADW3O9wk7Zk5OS3w5QDof_5NkshCQfK82z3ezudTCSM55Bi9KnnJMKMZ3r2EKmG4Q== --bucket telegraf --skip-verify /backups/influxdb2_backup

2020-11-11T16:02:33.502682Z info Backing up KV store {"log_id": "0QQ0Y~cW000", "path": "20201111T160233Z.bolt"} 2020-11-11T16:02:33.508030Z info Resources opened {"log_id": "0QQ0Y~cW000", "path": "/backups/influxdb2_backup/20201111T160233Z.bolt"}

Volumes for persisting data, see https://docs.docker.com/engine/admin/volumes/volumes/

2020-11-11T16:02:33.508355Z info Backing up organization {"log_id": "0QQ0Y~cW000", "id": "2b0bec00ff5d5fab", "name": "mnsmithuk"} 2020-11-11T16:02:33.508470Z info Backing up bucket {"log_id": "0QQ0Y~cW000", "id": "81694707ab03e09d", "name": "telegraf"} 2020-11-11T16:02:33.508495Z info Backing up shard {"log_id": "0QQ0Y~cW000", "id": 2, "path": "20201111T160233Z.s2.tar.gz"} 2020-11-11T16:03:00.775241Z info Writing manifest {"log_id": "0QQ0Y~cW000", "path": "20201111T160233Z.manifest"} 2020-11-11T16:03:00.775954Z info Backup complete {"log_id": "0QQ0Y~cW000"} root@08bc94041a0d:/# exit #

Actual behavior: influx backup --org-id 2b0bec00ff5d5fab --token Zz_Plltgu0MwAvTI8vnTkADW3O9wk7Zk5OS3w5QDof_5NkshCQfK82z3ezudTCSM55Bi9KnnJMKMZ3r2EKmG4Q== --bucket telegraf --skip-verify /backups/influxdb2_backup 2020-11-11T16:16:10.910266Z info Backing up KV store {"log_id": "0QQ1KtcW000", "path": "20201111T161610Z.bolt"} Error: Attempted to unmarshal error as JSON but failed: "invalid character 'C' looking for beginning of value": Client sent an HTTP request to an HTTPS server. Error: Attempted to unmarshal error as JSON but failed: "invalid character 'C' looking for beginning of value": Client sent an HTTP request to an HTTPS server. See 'influx backup -h' for help root@08bc94041a0d:/# exit exit Error: non zero exit code: 1: OCI runtime error #

Environment info: Linux 4.18.0-193.28.1.el8_2.x86_64 x86_64 CentOS 8.2.2004 podman 1.6.4 podman-compose 0.1.7dev InfluxDB 2.0.1 (git: 37cc047133) build_date: 2020-11-11T03:53:31Z

Config:

docker-compose.yml version: '3.8' networks: influxdb_net: driver: bridge

Volumes for persisting data, see https://docs.docker.com/engine/admin/volumes/volumes/

volumes: influxdb2_data: driver: local

services:

Define an InfluxDB service

influxdb: image: quay.io/influxdb/influxdb:v2.0.1 container_name: influxdb2 volumes:

Mount for influxdb data directory

  - influxdb2_data:/var/lib/influxdb2
  - ./etc/ssl/certs/influxdb2-selfsigned-cert.pem:/etc/ssl/certs/influxdb2-selfsigned-cert.pem
  - ./etc/ssl/private/influxdb2-selfsigned-key.pem:/etc/ssl/private/influxdb2-selfsigned-key.pem
command: influxd run --bolt-path /var/lib/influxdb2/influxd.bolt --engine-path /var/lib/influxdb2/engine --store bolt --tls-cert /etc/ssl/certs/influxdb2-selfsigned-cert.pem --tls-key /etc/ssl/private/influxdb2-selfsigned-key.pem --http-bind-address=:8086
ports:
  # The API for InfluxDB is served on port 8086
  - "8086:8086"
#restart: unless-stopped
networks:
  - influxdb_net

Logs:

Container Logs ts=2020-11-11T16:04:15.127542Z lvl=info msg="Welcome to InfluxDB" log_id=0QQ0eCal000 version=2.0.1 commit=37cc047133 build_date=2020-11-11T03:53:31Z ts=2020-11-11T16:04:15.128324Z lvl=info msg="Resources opened" log_id=0QQ0eCal000 service=bolt path=/var/lib/influxdb2/influxd.bolt ts=2020-11-11T16:04:15.133765Z lvl=info msg="Checking InfluxDB metadata for prior version." log_id=0QQ0eCal000 bolt_path=/var/lib/influxdb2/influxd.bolt ts=2020-11-11T16:04:15.133946Z lvl=info msg="Using data dir" log_id=0QQ0eCal000 service=storage-engine path=/var/lib/influxdb2/engine/data ts=2020-11-11T16:04:15.133962Z lvl=info msg="Compaction settings" log_id=0QQ0eCal000 service=storage-engine max_concurrent_compactions=4 throughput_bytes_per_second=50331648 throughput_bytes_per_second_burst=50331648 ts=2020-11-11T16:04:15.133971Z lvl=info msg="Open store (start)" log_id=0QQ0eCal000 service=storage-engine op_name=tsdb_open op_event=start ts=2020-11-11T16:04:15.160106Z lvl=info msg="Opened shard" log_id=0QQ0eCal000 service=storage-engine op_name=tsdb_open index_version=tsi1 path=/var/lib/influxdb2/engine/data/27331dbc65036c95/autogen/4 duration=24.904ms ts=2020-11-11T16:04:15.163023Z lvl=info msg="Opened shard" log_id=0QQ0eCal000 service=storage-engine op_name=tsdb_open index_version=tsi1 path=/var/lib/influxdb2/engine/data/af89360fbdf66960/autogen/3 duration=14.214ms ts=2020-11-11T16:04:15.163725Z lvl=info msg="Opened shard" log_id=0QQ0eCal000 service=storage-engine op_name=tsdb_open index_version=tsi1 path=/var/lib/influxdb2/engine/data/27331dbc65036c95/autogen/1 duration=28.562ms ts=2020-11-11T16:04:15.172500Z lvl=info msg="Opened shard" log_id=0QQ0eCal000 service=storage-engine op_name=tsdb_open index_version=tsi1 path=/var/lib/influxdb2/engine/data/af89360fbdf66960/autogen/5 duration=23.711ms ts=2020-11-11T16:04:15.183923Z lvl=info msg="Opened shard" log_id=0QQ0eCal000 service=storage-engine op_name=tsdb_open index_version=tsi1 path=/var/lib/influxdb2/engine/data/81694707ab03e09d/autogen/2 duration=38.431ms ts=2020-11-11T16:04:15.252438Z lvl=info msg="Opened shard" log_id=0QQ0eCal000 service=storage-engine op_name=tsdb_open index_version=tsi1 path=/var/lib/influxdb2/engine/data/27331dbc65036c95/autogen/6 duration=117.125ms ts=2020-11-11T16:04:15.433245Z lvl=info msg="Opened shard" log_id=0QQ0eCal000 service=storage-engine op_name=tsdb_open index_version=tsi1 path=/var/lib/influxdb2/engine/data/af89360fbdf66960/autogen/7 duration=286.052ms ts=2020-11-11T16:04:15.433450Z lvl=info msg="Open store (end)" log_id=0QQ0eCal000 service=storage-engine op_name=tsdb_open op_event=end op_elapsed=299.479ms ts=2020-11-11T16:04:15.433467Z lvl=info msg="Starting retention policy enforcement service" log_id=0QQ0eCal000 service=retention check_interval=30m ts=2020-11-11T16:04:15.433483Z lvl=info msg="Starting precreation service" log_id=0QQ0eCal000 service=shard-precreation check_interval=10m advance_period=30m ts=2020-11-11T16:04:15.433522Z lvl=info msg="Starting query controller" log_id=0QQ0eCal000 service=storage-reads concurrency_quota=10 initial_memory_bytes_quota_per_query=9223372036854775807 memory_bytes_quota_per_query=9223372036854775807 max_memory_bytes=0 queue_size=10 ts=2020-11-11T16:04:15.438095Z lvl=info msg="Configuring InfluxQL statement executor (zeros indicate unlimited)." log_id=0QQ0eCal000 max_select_point=0 max_select_series=0 max_select_buckets=0 ts=2020-11-11T16:04:15.734171Z lvl=info msg=Starting log_id=0QQ0eCal000 service=telemetry interval=8h ts=2020-11-11T16:04:15.734208Z lvl=info msg=Listening log_id=0QQ0eCal000 transport=https addr=:8086 port=8086

There were no releated logs in /var/log/messages

benbjohnson commented 4 years ago

Thanks for the bug report. We have a fix here: https://github.com/influxdata/influxdb/pull/19991