Service Failed to Run

[Coelhomatias]

Steps to reproduce: List the minimal actions needed to reproduce the behavior.

1. sudo systemctl start influxdb

Expected behavior: Service to start

Actual behavior: Service fails and tries to reconnect

Environment info:

• System info: Linux 4.19.97-v7+ armv7l (Raspberry Pi 3 B+)
• InfluxDB version: InfluxDB v1.7.10 (git: 1.7 f46f63d4e2d9684a2dd716594ab609ccd32f0a5b)

Config:

Added bind-address = "127.0.0.1:8090" to /ect/influxdb/influxdb.conf

Logs:

Apr 16 18:44:31 raspberrypi systemd[1]: influxdb.service: Service RestartSec=100ms expired, scheduling restart.
Apr 16 18:44:31 raspberrypi systemd[1]: influxdb.service: Scheduled restart job, restart counter is at 23.
Apr 16 18:44:31 raspberrypi systemd[1]: Stopped InfluxDB is an open-source, distributed, time series database.
Apr 16 18:44:31 raspberrypi systemd[1]: Started InfluxDB is an open-source, distributed, time series database.
Apr 16 18:44:56 raspberrypi influxd[1412]: ts=2020-04-16T17:44:56.474125Z lvl=info msg="InfluxDB starting" log_id=0MD1LVbW000 version=1.7.10 branch=1.7 commit=f46f63d4e2d9684a2dd716594ab609ccd32f0a5b
Apr 16 18:44:56 raspberrypi influxd[1412]: ts=2020-04-16T17:44:56.474546Z lvl=info msg="Go runtime" log_id=0MD1LVbW000 version=go1.12.6 maxprocs=4
Apr 16 18:44:56 raspberrypi influxd[1412]: ts=2020-04-16T17:44:56.593172Z lvl=info msg="Using data dir" log_id=0MD1LVbW000 service=store path=/var/lib/influxdb/data
Apr 16 18:44:56 raspberrypi influxd[1412]: ts=2020-04-16T17:44:56.594974Z lvl=info msg="Compaction settings" log_id=0MD1LVbW000 service=store max_concurrent_compactions=2 throughput_bytes_per_second=50331648 throughput_bytes_per_second_burst=50331648
Apr 16 18:44:56 raspberrypi influxd[1412]: ts=2020-04-16T17:44:56.596378Z lvl=info msg="Open store (start)" log_id=0MD1LVbW000 service=store trace_id=0MD1LW50000 op_name=tsdb_open op_event=start
Apr 16 18:44:56 raspberrypi influxd[1412]: ts=2020-04-16T17:44:56.598220Z lvl=info msg="Open store (end)" log_id=0MD1LVbW000 service=store trace_id=0MD1LW50000 op_name=tsdb_open op_event=end op_elapsed=1.868ms
Apr 16 18:44:56 raspberrypi influxd[1412]: run: open server: open tsdb store: mkdir /var/lib/influxdb/data/_internal/_series: permission denied
Apr 16 18:44:56 raspberrypi systemd[1]: influxdb.service: Main process exited, code=exited, status=1/FAILURE
Apr 16 18:44:56 raspberrypi systemd[1]: influxdb.service: Failed with result 'exit-code'.
Apr 16 18:44:56 raspberrypi systemd[1]: influxdb.service: Service RestartSec=100ms expired, scheduling restart.
Apr 16 18:44:56 raspberrypi systemd[1]: influxdb.service: Scheduled restart job, restart counter is at 24.
Apr 16 18:44:56 raspberrypi systemd[1]: Stopped InfluxDB is an open-source, distributed, time series database.
Apr 16 18:44:56 raspberrypi systemd[1]: Started InfluxDB is an open-source, distributed, time series database.
Apr 16 18:45:23 raspberrypi influxd[1435]: ts=2020-04-16T17:45:23.468029Z lvl=info msg="InfluxDB starting" log_id=0MD1N92l000 version=1.7.10 branch=1.7 commit=f46f63d4e2d9684a2dd716594ab609ccd32f0a5b
Apr 16 18:45:23 raspberrypi influxd[1435]: ts=2020-04-16T17:45:23.468190Z lvl=info msg="Go runtime" log_id=0MD1N92l000 version=go1.12.6 maxprocs=4
Apr 16 18:45:23 raspberrypi influxd[1435]: ts=2020-04-16T17:45:23.584014Z lvl=info msg="Using data dir" log_id=0MD1N92l000 service=store path=/var/lib/influxdb/data
Apr 16 18:45:23 raspberrypi influxd[1435]: ts=2020-04-16T17:45:23.584199Z lvl=info msg="Compaction settings" log_id=0MD1N92l000 service=store max_concurrent_compactions=2 throughput_bytes_per_second=50331648 throughput_bytes_per_second_burst=50331648
Apr 16 18:45:23 raspberrypi influxd[1435]: ts=2020-04-16T17:45:23.584262Z lvl=info msg="Open store (start)" log_id=0MD1N92l000 service=store trace_id=0MD1N9W0000 op_name=tsdb_open op_event=start
Apr 16 18:45:23 raspberrypi influxd[1435]: ts=2020-04-16T17:45:23.584588Z lvl=info msg="Open store (end)" log_id=0MD1N92l000 service=store trace_id=0MD1N9W0000 op_name=tsdb_open op_event=end op_elapsed=0.332ms
Apr 16 18:45:23 raspberrypi influxd[1435]: run: open server: open tsdb store: mkdir /var/lib/influxdb/data/_internal/_series: permission denied
Apr 16 18:45:23 raspberrypi systemd[1]: influxdb.service: Main process exited, code=exited, status=1/FAILURE
Apr 16 18:45:23 raspberrypi systemd[1]: influxdb.service: Failed with result 'exit-code'.
Apr 16 18:45:23 raspberrypi systemd[1]: influxdb.service: Service RestartSec=100ms expired, scheduling restart.

[Coelhomatias]

This happened when I would run sudo influxd without sudo, so I suspect it is something about permissions, I tried giving the influxdb user root privileges but didn't work

[glinton]

Based on the logs above, it is failing to start because the user running influxdb doesn't have permission to create the required directory.

mkdir /var/lib/influxdb/data/_internal/_series: permission denied

How did you install influxdb and did you create your own systemd service definition?

In the meantime, you should be able to workaround those permission issues by chowning the directories in question to the user that is defined to run influxd. I believe you can find that by running systemctl show influxdb --no-pager | grep 'UID=' If that shows up as [not set], you can check the systemd service definition file to see if it's clear what user it's started as. The file can be found by running systemctl status influxdb --no-pager | grep -i loaded. To set the proper permissions on the directory in question, once you've found what user you should change ownership to, run chown -R USER_YOU_FOUND /var/lib/influxdb

I'm very curious as to how influxdb was installed so we can look into fixing the installer if needed. I'm going to leave this open until we can confirm it's not a package/installer issue.

[Coelhomatias]

I installed using the installation process that can be found on the documentation. I managed to get a workaround by assigning the influxdb user as a root user. I'll be sure to try out your method as soon as I can. Thank you for the fast response!!