search

mimugmail / opn-repo

OPNsense repo by mimugmail
Other
369 stars 28 forks source link

Speedtest not working at all #177

Open Axsgranted opened 1 year ago

Axsgranted commented 1 year ago

Installed the plugin fine, but couldn't get a list of speedtest servers. No error just hangs forever on fetching the list of servers. Decided to swith to Ookla and servers did show up. When I attempted to run a report I got the following:

[31-Aug-2023 13:10:44 America/Phoenix] Phalcon\Mvc\Dispatcher\Exception: OPNsense\Speedtest\UndefinedController handler class cannot be loaded in /usr/local/opnsense/www/index.php:70 Stack trace:

0 [internal function]: Phalcon\Mvc\Dispatcher->throwDispatchException('OPNsense\Speedt...', 2)

1 [internal function]: Phalcon\Dispatcher\AbstractDispatcher->dispatch()

2 /usr/local/opnsense/www/index.php(70): Phalcon\Mvc\Application->handle('/ui/speedtest/u...')

3 {main}

Machine Specs and Version: OPNsense 23.7.3 running on: Dell Optiplex 3050 Intel I5-7600 @ 3.5Ghz (4 Cores) Intel I350-T4 Nic 8G DDR4 256G SSD

mihakralj commented 1 year ago

Check /usr/local/opnsense/scripts/OPNsense/speedtest directory and see if speedtest.csv is in there. While you are in that directory, let's try some basic troubleshooting:

python3 opn_speedtest.py list should generate a list of closest speedtest servers python3 opn_speedtest.py version should list the installed speedtest (in your case Ookla) python3 opn_speedtest.py 0 should run a test on auto-selected server python3 opn_speedtest.py log should list the content of speedtest.csv file (if it exists) python3 opn_speedtest.py stat generates stats from .csv file (if it exists)

Axsgranted commented 1 year ago

Speedtest.csv file is present.

I receive the following trying to run any command {"version": "none", "message": "No speedtest package installed"}

Before you ask, it is installed and I do see the script within the directory. When I try to install the cli via the GUI that's when it hangs on "fetching available Speedtest servers..." and never stops.

image

Axsgranted commented 1 year ago

Ok did some more testing.

speedtest-cli doesn't work in any scenario If i refresh the page and select Ookla, I can do "socket" testing and I did get a result. However if I attempt to switch to "http speedtest" it complains the module is not installed AND breaks the ookla test. I.e. if I run a test again, ookla test stops immediately and reports nothing.

Axsgranted commented 1 year ago

Anything else I can try?

mimugmail commented 1 year ago

I pushed the update now, please remove and reinstall and try again

Axsgranted commented 1 year ago

Uninstalled and re-installed and was presenting with an instant list of servers to choose from. i did not have to choose to install speedtest-cli or ookla. It just showed list and run test.

If this is by design, then it worked perfectly :)

mihakralj commented 1 year ago

Yes, by design. It will default to whatever is already installed and you can switch the speedtest from Python-based (CLI) to binary from Ookla - and back.

Axsgranted commented 1 year ago

I saw no options to switch in the GUI. Is that a command line option?

mihakralj commented 1 year ago

did you click the Help switch on the Speedtest plugin page top right?

Axsgranted commented 1 year ago

I did not, I see it now ;) "HELP" in general wasn't an intuitive thing for me to think of when changing a config ;) FYI I switched to speedtest HTTP and the same issue occurs. no list if servers.

mihakralj commented 1 year ago

I was young and stupid when I wrote the plugin, didn't really know what I am doing. That was my first adventure in making something for OPNsense and I am surprised that the trash code I created lasted that long and the plugin is still working. Well, I am old and stupid now, but the plugin is opensource on Michael's repo (right here) - and all improvements are very welcome. The change you want needs to be done somewhere in https://github.com/mimugmail/opn-repo/blob/main/net-mgmt/speedtest-community/src/opnsense/mvc/app/views/OPNsense/Speedtest/index.volt

Axsgranted commented 1 year ago

Lol all good. While i'm also no longer young, I haven't touched any real code in many years. I'm afraid I'd do more harm than good ;)

I mean at the end of the day, the sockets test works, and OK with that :)

Axsgranted commented 1 year ago

I could try installing the speedtest-cli manually? I think there are some threads on it already. EDIT: Apparently it's already installed, well at least py39-speedtest-cli-2.1.3 is.

Axsgranted commented 1 year ago

Another update: not sure what happened but after closing the browser and running the previous commands you suggested, when I went back in to the GUI it's working?? however most of the servers are returning "Speedtest server id xxxxx not recognized.", but some DO work.

mihakralj commented 1 year ago

so, as you noticed, there are two speedtest implementations - the python one and the binary one. They can't co-exist as then have the same name; it is the one or the other.

I created a simple bash script install_speedtest.sh (in /usr/local/opnsense/scripts/OPNsense/speedtest) that takes one of three arguments:

When either package is installed, it has the same name: speedtest with slightly different parameters structure. And, oddly, some server IDs work only under Ookla binary package. My assumption is that old speedtest servers accept both http and socket requests, but new/upgraded speedtest servers accept socket calls only (therefore Ookla binary works in more cases than Python version)

Reloading the Speedtest webpage should cast a new list of closest servers. You can do that manually too:

Ookla binary: speedtest --accept-license --accept-gdpr --servers Python CLI: speedtest --list

in my case, see the difference in listed servers: Ookla:

    ID  Name                           Location             Country
==============================================================================
 51693  Ziply Fiber                    Bothell, WA          United States
 35180  Ziply Fiber                    Seattle, WA          United States
  8864  CenturyLink                    Seattle, WA          United States
 37495  Netprotect                     Seattle, WA          United States
 12192  Nitel                          Seattle, WA          United States
  6199  Wowrack                        Seattle, WA          United States
 49421  Lumen Technologies             Seattle, WA          United States
 22168  Whitesky Communications LLC    Seattle, WA          United States
 50679  Misaka Network, Inc.           Seattle, WA          United States
  1782  Comcast                        Seattle, WA          United States

Python CLI:

35180) Ziply Fiber (Seattle, WA, United States) [2269.78 km]
 8864) CenturyLink (Seattle, WA, United States) [2269.78 km]
49421) Lumen Technologies (Seattle, WA, United States) [2269.78 km]
22168) Whitesky Communications LLC (Seattle, WA, United States) [2269.78 km]
50679) Misaka Network, Inc. (Seattle, WA, United States) [2269.78 km]
 6057) Accretive Networks (Seattle, WA, United States) [2269.78 km]
18532) Wave (Port Orchard, WA, United States) [2288.81 km]
54406) Whidbey Telecom (Langley, WA, United States) [2291.76 km]
40251) iFiber Communications (Shelton, WA, United States) [2310.44 km]
31110) City of Anacortes (Anacortes, WA, United States) [2325.07 km]
mihakralj commented 1 year ago

Apparently it's already installed, well at least py39-speedtest-cli-2.1.3 is

The nightmare of slippery names is a problem with both packages - Python and Binary. Therefore I updated the install_speedtest.sh to try to catch the name of the latest available package:

Axsgranted commented 1 year ago

Apparently it's already installed, well at least py39-speedtest-cli-2.1.3 is

The nightmare of slippery names is a problem with both packages - Python and Binary. Therefore I updated the install_speedtest.sh to try to catch the name of the latest available package:

  • for Python package (which is in OPNSense repo): PYTHON_PKG_NAME=$(pkg info -x 'python3*' | grep 'python3' | sed -n 's/^python\([0-9]*\).*/py\1-speedtest-cli/p')
  • for Binary version (not in any standard repos, need direct URL): OOKLA_URL=$(curl -s "https://www.speedtest.net/apps/cli" | xmllint --html --xpath "string(//*[@id='freebsd']/pre/code[7]/text())" - 2>/dev/null | awk -F'"' '{print $2}') (you can echo $PYTHON_PKG_NAME and echo $OOKLA_URL to see the calculated current package name/url)

Yup I was actually just checking this but I oddly had to change execute permissions to get it to run, which begs the question, without looking at the repo, how does this run in the first place? do you copy, run, delete or something? FYI the package name and URL both enumerated perfectly.

Also curious, if new/updated servers are all socket only why not just have the one binary option going forward and simplify?

mihakralj commented 1 year ago

@mimugmail do we have the exec flag set in the latest speedtest package? I cannot see the settings in the Github source tree - we should check that there is a chmod +x on all .sh commands in the package.

mihakralj commented 1 year ago

if new/updated servers are all socket only why not just have the one binary option going forward and simplify?

Ookla binary is not open source and Python CLI is. OPNsense prefers OSS whenever possible. So, default is OSS speedtest with an option to replace it with a (proprietary) binary.

mimugmail commented 1 year ago

You can add a PR with a Post install file and chmod?

mihakralj commented 1 year ago

I tested around and packager (pkg create) honors flags on the file as it packages it into the archive; we just need to set the flag right in the repo; will submit the PR for it.

BTW, I found out that I can package the .txz (or .pkg) for FreeBSD on Linux as well, using bsdtar:

The -s is a substitution option that will remove ./ at the beginning of filenames and make paths absolute, allowing archive to be extracted to the absolute path.

mimugmail commented 1 year ago

Why not create a file like here and adding a chmod

mihakralj commented 1 year ago

that works too, but it is unnecessary if we set the +x flag on the file before packing it. Let me play with it.

mihakralj commented 1 year ago

https://github.com/mimugmail/opn-repo/pull/180 has +x flag set on .sh and .py files required for this plugin. Let's repackage and see if FreeBSD documentation is right...

mihakralj commented 1 year ago

this issue is ready to be closed.