search

barrycarey / Speedtest-for-InfluxDB-and-Grafana

A tool to run periodic speedtests and save them in InfluxDB for graphing in Grafana
146 stars 73 forks source link

Latest Docker Image fails immediately on start #26

Open n1nj4888 opened 5 years ago

n1nj4888 commented 5 years ago

I just updated the atribe/Speedtest-for-InfluxDB-and-Grafana image to the latest version and the container restarts because of the following errors immediately after starting the container - This didn't happen with the previous container/image so not sure what has changed here in this latest build?

Loading Configuration File config.ini Configuration Successfully Loaded 2019-02-11 01:29:55,627 - DEBUG: Testing connection to InfluxDb using provided credentials

There are no other indicators as to why to container dies and it dies that quickly after starting that I can't seem to get into the container to find out why?

PS: I'm running the docker atribe/Speedtest-for-InfluxDB-and-Grafana:latest on Synology

Thanks!

cyablo commented 5 years ago

Same here without container. Script simply finishes after "Configuration Successfully Loaded". No other output, no error.

fanaticDavid commented 5 years ago

I upgraded the image 2 days ago (also on a Synology NAS), and I am encountering exactly the same issue: the container is stuck restarting. Here's a sample from the logs, obtained using Portainer:

Loading Configuration File config.ini
Configuration Successfully Loaded
2019-02-13 22:42:22,980 - DEBUG: Testing connection to InfluxDb using provided credentials
Loading Configuration File config.ini
Configuration Successfully Loaded
2019-02-13 22:42:50,590 - DEBUG: Testing connection to InfluxDb using provided credentials
Loading Configuration File config.ini
Configuration Successfully Loaded
2019-02-13 22:43:44,746 - DEBUG: Testing connection to InfluxDb using provided credentials
Loading Configuration File config.ini
Configuration Successfully Loaded
2019-02-13 22:44:47,671 - DEBUG: Testing connection to InfluxDb using provided credentials

If the speed test is even run at all, the results are certainly not being saved to InfluxDB.

barrycarey commented 5 years ago

I don't maintain the Docker image currently. If I have time this weekend I'll setup an officially supported on.

billimek commented 5 years ago

Not sure why this is causing an issue for you two. For what it's worth, I'm running the atribe/speedtest-for-influxdb-and-grafana:latest docker image chart definition here and the config file at runtime looks like this:

/src # cat config.ini
[GENERAL]
Delay = 7200
[INFLUXDB]
Address = influxdb-influxdb
Port = 8086
Database = speedtests
Username =
Password =
Verify_SSL = False
[SPEEDTEST]
Server =
[LOGGING]
Level = debug

sample log output is:

Loading Configuration File config.ini
Configuration Successfully Loaded
2019-02-15 14:06:39,683 - DEBUG: Testing connection to InfluxDb using provided credentials
2019-02-15 14:06:39,816 - DEBUG: Successful connection to InfluxDb
2019-02-15 14:06:39,816 - INFO: Starting Speed Test For Server None
2019-02-15 14:06:39,892 - DEBUG: Setting up SpeedTest.net client
2019-02-15 14:06:40,505 - DEBUG: Picking the closest server
2019-02-15 14:06:41,207 - INFO: Selected Server 1767 in Atlanta, GA
2019-02-15 14:06:41,208 - INFO: Starting download test
2019-02-15 14:06:51,237 - INFO: Starting upload test
2019-02-15 14:07:01,860 - DEBUG: [{'measurement': 'speed_test_results', 'fields': {'download': 269215505.17997843, 'upload': 23655377.193265762, 'ping': 18.512}, 'tags': {'server': '1767'}}]
2019-02-15 14:07:01,886 - DEBUG: Data written to InfluxDB
2019-02-15 14:07:01,887 - INFO: Download: 269.22Mbps - Upload: 23.66Mbps - Latency: 18.512ms
2019-02-15 14:07:01,887 - INFO: Waiting 7200 seconds until next test
cyablo commented 5 years ago

It's not only the Docker Container. The latest bare Script also does not work on my Debian Stretch Machine:

Loading Configuration File config.ini Configuration Successfully Loaded 2019-02-18 19:34:43,188 - DEBUG: Testing connection to InfluxDb using provided credentials root@cybstretch1:~/Speedtest-for-InfluxDB-and-Grafana#

barrycarey commented 5 years ago

@cyablo I just did a pull straight from the Repo on Ubuntu and it worked no issue. Can you post your config? Just leave out username/password stuff.

barrycarey commented 5 years ago

Ok, pretty sure I know what's going on. If it fails to connect to InfluxDB it was failing silently with a sys.exit(). Looks like each time this happens the container restarts the script so it just loops.

I just updated the repo to output an error if it fails to connect to Influx.

For any of you having this issue, ensure your InfluxDB connection details are correct.

atribe commented 5 years ago

I just pulled that change into the docker image and it is rebuilding right now.

cyablo commented 5 years ago

You are right:

2019-02-18 19:59:33,803 - CRITICAL: Failed to connect to InfluxDB for unknown reason

But my InfluxDB Credentials are 100% correct. I even did reset the password to be sure and I'm using the same credentials in Grafana. Here is my config:

[GENERAL]

Delay = 300

[INFLUXDB] Address = localhost Port = 8086 Database = speedtests Username = speedtests Password = ** Verify_SSL = False

[SPEEDTEST]

Leave blank to auto pick server

Server =

[LOGGING]

Valid Options: critical, error, warning, info, debug

Level = debug

barrycarey commented 5 years ago

Are you influx with or without SSL?

cyablo commented 5 years ago

I'm not using SSL on Influx.

barrycarey commented 5 years ago

If you have python on your machine pip install influxdb and then try running this (either save as .py or do it in the Python console):

influx = InfluxDBClient( '111.111.111.111, 8086, database=DATABASE, ssl=False, verify_ssl=False, username=Username, password=Password, timeout=5 )

influx.get_list_users()

Trying to figure out if this is an issue with my script or just a failure connect to Influx in general.

cyablo commented 5 years ago

I have a few own Scripts, FHEM & LibreNMS pushing data to Influx without Problems. An older Version of your Script was also working fine.

THERE WAS CRAP HERE

cyablo commented 5 years ago

My bad, forgot to add '' to my database name, username etc...

influxdb.exceptions.InfluxDBClientError: 403: {"error":"error authorizing query: speedtests not authorized to execute statement 'SHOW USERS', requires admin privilege"}

barrycarey commented 5 years ago

Do you have restricted permissions on that user?

Currently, to test if the Influx Connection is working I'm sending a show users command. If the command throws and exception the script exists. I didn't do this prior to the major updates I did recently which would explain why it worked before.

I never explored what level of permissions were required for that command. It might require root. You may need to use a user with more permissions until I can explore a better way to verify the Influx credentials.

cyablo commented 5 years ago

Sure, the user only has permission to read and write to it's own DB.

I just commented out the lines to test the connection, it's working now and I'll take it as a workaround.

barrycarey commented 5 years ago

Glad to hear that worked. Admittedly not an ideal work around. When I have some time I'll see if I can come up with a better way to verify the credentials.

n1nj4888 commented 5 years ago

Thanks for looking into this - I can confirm I also use a user which only has read/write to the specific speedtest DB in influxdb so assume this is the cause of the issue. Rather than granting that user admin rights, I’d also like to comment out the access test if possible as a workaround so am keen to understand:

(1) Which files / lines need commenting out? (2) How are you commenting these lines out since I only expose the container’s /src (the config files, no code) at present and cant get into the container quick enough after starting it to make the amendments before the container errors and stops?

Thanks!

barrycarey commented 5 years ago

It's line 41 in InfluxdbSpeedtest.py that's causing the issue.

The easiest solution would be using a user with root permissions temporary. I'll get it fixed this weekend.

fanaticDavid commented 5 years ago

In the Home Assistant project, they recently introduced a similar bug. They fixed it by using the following query: SHOW SERIES LIMIT 1; I'm not sure if that would work in this case, but I just thought I'd share :wink:

barrycarey commented 5 years ago

Nice! Thanks for the heads up. I'll mess with it this weekend.

n1nj4888 commented 5 years ago

Hi @barrycarey - Were you able to fix the "show users" error stopping the container?

n1nj4888 commented 5 years ago

Hi @barrycarey - Do you plan to release a fix for this? The only way i can get the docker to run currently is to manually comment out the "show users" test in the script inside the container...

n1nj4888 commented 5 years ago

Hi @barrycarey - Just checking in on whether you're still planning to fix this to save those of us with user permissions in InfluxDB from having to manually edit the line 41 of InfluxdbSpeedtest.py that's causing the issue?

Thanks!

douglasg14b commented 4 years ago

Welp, this is a shame....

My user only has access to the db, and not admin rights, so of course that fails.

danielo515 commented 4 years ago

So, it is just a one-line change right? Would you will to do it? I will just fork the project and do it myself, but I'm using a helm chart that uses this image, so many levels of indirection...