search

danopstech / speedtest_exporter

🏎️ Prometheus exporter that runs speedtest and exposes results
GNU General Public License v3.0
54 stars 19 forks source link

Accuracy of this speedtest code is unknown #10

Open virtuallynathan opened 3 years ago

virtuallynathan commented 3 years ago

Speedtests can be pretty all over the place, I haven't had a chance to fully evaluate this one, but with a few initial tests it does not appear to be very accurate. I think for maximum accuracy, just exec'ing the official speedtest.net binary would probably be best. It will output json and other formats.

Great work overall on this project, I just set it up.

virtuallynathan commented 3 years ago

Target Server: [35180] 17.14km Seattle, WA (United States) by Ziply Fiber Latency: 2.483086ms Download Test: . Upload Test: .

Download: 733.27 Mbit/s Upload: 866.26 Mbit/s

vs: 

Server: Ziply Fiber - Seattle, WA (id = 35180)
    ISP: Ziply Fiber
Latency:     1.95 ms   (0.16 ms jitter)

Download: 919.35 Mbps (data used: 450.2 MB) Upload: 934.45 Mbps (data used: 420.9 MB)

dwillcocks commented 3 years ago

Totally agree, every speed test shows different results, but by using the same one yields some kind of consistency. For most users (me included at the point of writing it) it only needed to show tends. For me it was to tend the speed changes as Starlink release software changes to their hardware.

I have not looked deeply into it, but sometimes it feels Docker is constraining the speedtest, but then I'm running it on Mac so Docker is inside a VM aswell.

I did POC using Speedtests.net CLI binary but it started to get messy (cant remember exactly but I ended up with external dependencies) then a bigger final image size, not running on distroless/static, so included vulnerabilities. and needing to accept end user agreements.

I may try again now I have my external Starlink monitoring up and running, I can iterate on the exporters and work on accuracy.

😉 If only Netflix would release more information around fast.com

virtuallynathan commented 3 years ago

That makes sense, I may take a look at writing some code to run the speedtest-cli and see how messy it becomes. I think i'll also poke around with the Go speedtest impl you are using and see if any tweaks make it more accurate.

If only... haha.

dwillcocks commented 3 years ago

If only... haha.

fast_exporter maybe!?!

CharlesGodwin commented 3 years ago

I agree with accuracy problem. I am currently getting negative results. These make it unusable.

curl http://localhost:9092/metrics
# HELP speedtest_download_speed_Bps Last download speedtest result
# TYPE speedtest_download_speed_Bps gauge
speedtest_download_speed_Bps{distance="1.387210",server_country="Canada",server_id="16754",server_lat="45.5017",server_lon="-73.5673",server_name="Montreal, QC",test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb",user_ip="135.129.117.112",user_isp="Starlink",user_lat="45.4995",user_lon="-73.5848"} 7.87970036405554e+07
# HELP speedtest_latency_seconds Measured latency on last speed test
# TYPE speedtest_latency_seconds gauge
speedtest_latency_seconds{distance="1.387210",server_country="Canada",server_id="16754",server_lat="45.5017",server_lon="-73.5673",server_name="Montreal, QC",test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb",user_ip="135.129.117.112",user_isp="Starlink",user_lat="45.4995",user_lon="-73.5848"} 2.058390779
# HELP speedtest_scrape_duration_seconds Time to preform last speed test
# TYPE speedtest_scrape_duration_seconds gauge
speedtest_scrape_duration_seconds{test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb"} 25.850604899
# HELP speedtest_up Was the last speedtest successful.
# TYPE speedtest_up gauge
speedtest_up{test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb"} 1
# HELP speedtest_upload_speed_Bps Last upload speedtest result
# TYPE speedtest_upload_speed_Bps gauge
speedtest_upload_speed_Bps{distance="1.387210",server_country="Canada",server_id="16754",server_lat="45.5017",server_lon="-73.5673",server_name="Montreal, QC",test_uuid="925c87e8-43bf-4aa7-b056-c40da92630bb",user_ip="135.129.117.112",user_isp="Starlink",user_lat="45.4995",user_lon="-73.5848"} -2.2512346881061174e+07
CharlesGodwin commented 3 years ago

Is there any way to encourage you to deal with this. I am currently getting persistent negative values for uploads. I'd look inot it myself bu I have no experience with Prometheus or Go.

Thank you

keyaertc commented 2 years ago

Same for me, unfortunately, the results are inaccurate. Any we can do to help ?

jbartak commented 2 years ago

Same for me, constant negative upload speeds