Speedtest command appears to be broken

[gcastle2007]

          I made instruction:

## If migrating from prior bintray install instructions please first...
# sudo rm /etc/apt/sources.list.d/speedtest.list
# sudo apt-get update
# sudo apt-get remove speedtest
## Other non-official binaries will conflict with Speedtest CLI
# Example how to remove using apt-get
# sudo apt-get remove speedtest-cli
sudo apt-get install curl
curl -s https://packagecloud.io/install/repositories/ookla/speedtest-cli/script.deb.sh | sudo bash
sudo apt-get install speedtest

License accepted:

Do you accept the license? [type YES to accept]: YES
License acceptance recorded. Continuing.

I don't have process after command "speedtest" in bot:

wlanpi@wlanpi-359:~ $ ps -ax | grep speedtest
   4769 pts/0    S+     0:00 grep --color=auto speedtest
wlanpi@wlanpi-359:~ $

Originally posted by @gcastle2007 in https://github.com/WLAN-Pi/wlanpi-chat-bot/issues/46#issuecomment-2390532559

[gcastle2007]

Command speedtest | egrep... work correctly in console:

wlanpi@wlanpi-359:/opt/wlanpi-chat-bot/lib/python3.9/site-packages/chatbot/py_commands $ speedtest | egrep -w "Testing from|Download|Upload" | sed -r 's/Testing from.*?\(/My IP: /g; s/\)\.\.\.//g; s/Download/D/g; s/Upload/U/g; s/bit\/s/bps/g'
    D:   862.87 Mbps (data used: 1.4 GB)
      U:   895.67 Mbps (data used: 1.5 GB)
wlanpi@wlanpi-359:/opt/wlanpi-chat-bot/lib/python3.9/site-packages/chatbot/py_commands $

But speedtest-cli |egrep... have result beautefull:

wlanpi@wlanpi-359:/opt/wlanpi-chat-bot/lib/python3.9/site-packages/chatbot/py_commands $ speedtest-cli | egrep -w "Testing from|Download|Upload" | sed -r 's/Testing from.*?\(/My IP: /g; s/\)\.\.\.//g; s/Download/D/g; s/Upload/U/g; s/bit\/s/bps/g'
My IP: 82.140.69.78
D: 414.86 Mbps
U: 515.27 Mbps
wlanpi@wlanpi-359:/opt/wlanpi-chat-bot/lib/python3.9/site-packages/chatbot/py_commands $

But I don't see this command being executed from speedtest.py. The process with "speedtest" doesn't appear.

        speedtest_cmd = "speedtest | egrep -w \"Testing from|Download|Upload\" | sed -r 's/Testing from.*?\(/My IP: /g; s/\)\.\.\.//g; s/Download/D/g; s/Upload/U/g; s/bit\/s/bps/g'"

My WLANPI:

wlanpi@wlanpi-359:~ $ uname -a
Linux wlanpi-359 6.7.1-v8-wlanpi+ #1 SMP PREEMPT Mon Jan 22 17:15:38 UTC 2024 aarch64 GNU/Linux

wlanpi@wlanpi-359:~ $ wlanpi-model
Model:                WLAN Pi M4
Main board:           Mcuzone M4
USB Wi-Fi adapter:    0489:e0e0 Foxconn / Hon Hai Wireless_Device
M.2 Wi-Fi adapter:    MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless Network Adapter [14c3:0616]
Bluetooth adapter:    Built-in
wlanpi@wlanpi-359:~ $

[joshschmelzle]

Thanks for installing the binary and doing this testing (#46).

It's clear there are performance differences in speedtest-cli and speedtest from Ookla in your results posted above.

Possibly something changed in the Ookla binary between when Nigel wrote his wrapper code 3 years ago.

Needs further attention from someone.

[joshschmelzle]

Do you see still see 📈 Running speedtest...please wait ⏳ or did the behavior change?

[gcastle2007]

I found error. speedtest need to start with parameters:

speedtest --accept-license

I will made pull request. Please check.

Additional information from /var/log/syslog for this solution:

wlanpi@wlanpi-359:/var/run $ tail -f /var/log/syslog
Oct  4 14:13:56 wlanpi-359 chatbot[31418]: ==============================================================================
Oct  4 14:13:56 wlanpi-359 chatbot[31418]: You may only use this Speedtest software and information generated
Oct  4 14:13:56 wlanpi-359 chatbot[31418]: from it for personal, non-commercial use, through a command line
Oct  4 14:13:56 wlanpi-359 chatbot[31418]: interface on a personal computer. Your use of this software is subject
Oct  4 14:13:56 wlanpi-359 chatbot[31418]: to the End User License Agreement, Terms of Use and Privacy Policy at
Oct  4 14:13:56 wlanpi-359 chatbot[31418]: these URLs:
Oct  4 14:13:56 wlanpi-359 chatbot[31418]: #011https://www.speedtest.net/about/eula
Oct  4 14:13:56 wlanpi-359 chatbot[31418]: #011https://www.speedtest.net/about/terms
Oct  4 14:13:56 wlanpi-359 chatbot[31418]: #011https://www.speedtest.net/about/privacy
Oct  4 14:13:56 wlanpi-359 chatbot[31418]: ==============================================================================
**Oct  4 14:13:56 wlanpi-359 chatbot[31418]: To accept the message please run speedtest interactively or use the following:
Oct  4 14:13:56 wlanpi-359 chatbot[31418]:     speedtest --accept-license**
Oct  4 14:14:06 wlanpi-359 kernel: [ 2365.718117] [UFW BLOCK] IN=wlan0 OUT=eth0 MAC=4c:d5:77:b5:1e:f3:8c:1d:96:cd:c8:69:08:00 SRC=172.16.43.101 DST=149.154.167.222 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=27145 DF PROTO=TCP SPT=47186 DPT=443 WINDOW=64240 RES=0x00 SYN URGP=0
Oct  4 14:14:23 wlanpi-359 kernel: [ 2382.563286] [UFW BLOCK] IN=wlan0 OUT=eth0 MAC=4c:d5:77:b5:1e:f3:8c:1d:96:cd:c8:69:08:00 SRC=172.16.43.101 DST=208.67.222.222 LEN=63 TOS=0x00 PREC=0x00 TTL=127 ID=43606 PROTO=UDP SPT=51492 DPT=53 LEN=43

speedtest want to write info in to /home/wlanpi-chat-bot/.config/ookla/speedtest-cli.json but wlanpi not contains path /home/wlanpi-chat-bot

Oct  4 14:48:13 wlanpi-359 chatbot[34663]: ==============================================================================
Oct  4 14:48:13 wlanpi-359 chatbot[34663]: You may only use this Speedtest software and information generated
Oct  4 14:48:13 wlanpi-359 chatbot[34663]: from it for personal, non-commercial use, through a command line
Oct  4 14:48:13 wlanpi-359 chatbot[34663]: interface on a personal computer. Your use of this software is subject
Oct  4 14:48:13 wlanpi-359 chatbot[34663]: to the End User License Agreement, Terms of Use and Privacy Policy at
Oct  4 14:48:13 wlanpi-359 chatbot[34663]: these URLs:
Oct  4 14:48:13 wlanpi-359 chatbot[34663]: #011https://www.speedtest.net/about/eula
Oct  4 14:48:13 wlanpi-359 chatbot[34663]: #011https://www.speedtest.net/about/terms
Oct  4 14:48:13 wlanpi-359 chatbot[34663]: #011https://www.speedtest.net/about/privacy
Oct  4 14:48:13 wlanpi-359 chatbot[34663]: ==============================================================================
Oct  4 14:48:13 wlanpi-359 chatbot[34663]: License acceptance recorded. Continuing.
Oct  4 14:48:13 wlanpi-359 chatbot[34663]: [error] Failed to save settings: boost::filesystem::create_directories: Read-only file system [system:30]: "/home/wlanpi-chat-bot/.config/ookla", "/home/wlanpi-chat-bot"
Oct  4 14:48:13 wlanpi-359 chatbot[34663]: [error] Path used: /home/wlanpi-chat-bot/.config/ookla/speedtest-cli.json

wlanpi@wlanpi-359:/var/run $ ls -la /home/
total 12
drwxr-xr-x  3 root   root   4096 Feb 19  2024 .
drwxr-xr-x 18 root   root   4096 Feb 19  2024 ..
drwxr-xr-x  6 wlanpi wlanpi 4096 Oct  2 17:36 wlanpi

But speedtest and speedtest-cli have different result:

wlanpi@wlanpi-359:/var/run $ speedtest-cli
Retrieving speedtest.net configuration...
Testing from PJSC MegaFon (82.140.69.78)...
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by INETCOM LLC (Moscow) [634.98 km]: 20.377 ms
Testing download speed................................................................................
Download: 479.74 Mbit/s
Testing upload speed......................................................................................................
Upload: 396.44 Mbit/s
wlanpi@wlanpi-359:/var/run $ speedtest

   Speedtest by Ookla

      Server: RETN - Moscow (id: 22050)
         ISP: PJSC MegaFon
Idle Latency:    11.72 ms   (jitter: 0.15ms, low: 11.66ms, high: 11.98ms)
    Download:   913.22 Mbps (data used: 993.7 MB)
                 20.10 ms   (jitter: 3.21ms, low: 11.49ms, high: 242.97ms)
      Upload:   877.83 Mbps (data used: 976.5 MB)
                 12.69 ms   (jitter: 4.55ms, low: 11.68ms, high: 248.36ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/05d12b7e-84eb-4f28-a511-0b0f53c39793

[joshschmelzle]

As mentioned in PR #49, we absolutely cannot run with --accept-license as a default behavior. The user needs to read the EULA and accept them on their own.

[joshschmelzle]

The changes in your other PR #48 have been reverted in fa09ed0c0cf2b8572b38e0bc4dae66fe9d3e5e18.

[gcastle2007]

Please show me page in guide of wlanpi-chat-bot about information about license agreement of speedtest. I not found this information in guide.

Therefore, users don't know that this needs to be done. And also don't know that for the chat bot you need to create the home directory in home. All this leads to the fact that using speedtest is not possible. You can refer to various agreements for a long time, but make the solution work.

[joshschmelzle]

Ookla speedtest is not a WLAN Pi application. Thus, WLAN Pi cannot automatically accept the EULA on behalf of the user. Add changes to incorporate a mechanism for the user to accept the EULA.

[joshschmelzle]

The approach that I would approve in a PR would be the command to return an error if the EULA is not accepted, then instruct the user to review the EULA and run a separate command to accept the EULA which would also handle any other setup like having a home directory.

[gcastle2007]

You can describe all the necessary actions in your documentation. This may be a working solution.

[gcastle2007]

Okay. My version of solution. Will this work for you?

Wlanpi Chat Bot work on user wlanpi-chat-bot. Login to wlanpi on wlanpi user. User must start speedtest and accept license:

speedtest

User must create home directory for user wlanpi-chat-bot and other directory:

sudo mkdir -p /home/wlanpi-chat-bot/.config/ookla

User must copy file speedtest-cli.json to /home/wlanpi-chat-bot/.config/ookla

sudo cp .config/ookla/speedtest-cli.json /home/wlanpi-chat-bot/.config/ookla/

User must change owner for /home/wlanpi-chat-bot and other directory:

sudo chown -R wlanpi-chat-bot /home/wlanpi-chat-bot/

User must restart service whanpi-chat-bot:

sudo systemctl restart wlanpi-chat-bot.service

End.

We no longer see errors when running speedtest from Telegram.

What do you say? If yes, I can add it to the documentation tomorrow.