kizniche / Mycodo

An environmental monitoring and regulation system
http://kylegabriel.com/projects/
GNU General Public License v3.0
2.96k stars 494 forks source link

InfluxDB Starting Issues Due to Slow CPU #1312

Closed tledwar closed 1 year ago

tledwar commented 1 year ago

Mycodo Version: 8.15.8 Python Version: 3.9.2 (default, Mar 12 2021, 04:06:34) [GCC 10.2.1 20210110] Database Version: 16b28ef31b5b

During install, the DB installation process failed because the connection to the endpoint timed out.

Since the raspberry Pi (Zero W) CPU is not of great performance, the influxd process is not starting in a timely manner. Seems to be about 3 minutes from screen output/logs: 2023-06-05T22:29:22.281420Z info InfluxDB starting {"log_id": "0iFQi2tl000", "version": "1.8.10", "branch": "1.8", "commit": "688e697c51fd"} 2023-06-05T22:29:22.283797Z info Go runtime {"log_id": "0iFQi2tl000", "version": "go1.13.8", "maxprocs": 1} 2023-06-05T22:32:14.001843Z info Using data dir {"log_id": "0iFQi2tl000", "service": "store", "path": "/var/lib/influxdb/data"} 2023-06-05T22:32:14.005589Z info Compaction settings {"log_id": "0iFQi2tl000", "service": "store", "max_concurrent_compactions": 1, "throughput_bytes_per_second": 50331648, "throughput_bytes_per_second_burst": 50331648} 2023-06-05T22:32:14.018356Z info Open store (start) {"log_id": "0iFQi2tl000", "service": "store", "trace_id": "0iFQsXlW000", "op_name": "tsdb_open", "op_event": "start"} 2023-06-05T22:32:14.021796Z info Open store (end) {"log_id": "0iFQi2tl000", "service": "store", "trace_id": "0iFQsXlW000", "op_name": "tsdb_open", "op_event": "end", "op_elapsed": "3.471ms"} 2023-06-05T22:32:14.034703Z info Opened service {"log_id": "0iFQi2tl000", "service": "subscriber"}

To fix this issue, it seems I needed to increase the sleep time in /usr/lib/influxdb/scripts/influxd-systemd-start.sh. See below.

When I modified the value to 10. The install scripts from upgrade_commands.sh for the DB could successfully run. In addition, the DB now starts and restarts successfully and my endpoints work.

... additional content from file removed

set +e attempts=0 url="$PROTOCOL://$HOST:$PORT/health" result=$(curl -k -s -o /dev/null $url -w %{http_code}) while [ "${result:0:2}" != "20" ] && [ "${result:0:2}" != "40" ]; do attempts=$(($attempts+1)) echo "InfluxDB API unavailable after $attempts attempts..." sleep 10 result=$(curl -k -s -o /dev/null $url -w %{http_code}) done echo "InfluxDB started" set -e

... additional content from file removed

tledwar commented 1 year ago

As a follow up to this issue, modifying sleep was not the real issue. When you use systemctl to start the influxdb service, by default the timeout for systemctl is too small.

Therefore, a potential fix is for the Timeout to be modified in your delivery of /lib/systemd/system/influxdb.service.

I added the following to my file and was able to get the system working through reboots, backend restarts and such. TimeoutStartSec=10min

kizniche commented 1 year ago

This has been discussed numerous times in these issues. There is nothing the Mycodo project can do to change the influxdb init script, it is something you need to take up with influxdb.