bug: running quickget --list on macOS results grep errors

[rowdy-fms]

I confirm this bug has not already been reported

• [X] I have searched the issues and this bug has not been reported previously

Describe the bug When running quickget --list on macOS (Sonoma, on an M2 Mac Studio), it responds with error grep: invalid option -- P twice, then produces a long list of available operating systems and versions. Most of the entries in the list are correct, but some are nonsensical, sparkylinux in particular.

To Reproduce Steps to reproduce the behavior:

1. Run quickget --list on macOS.
2. Observe the two grep errors, and the list of nonsensical versions for sparkylinux.

Expected behavior A complete list of available operating systems with no grep errors and no nonsensical output.

Quickget output Run quickemu or quickemu and include the output of the failure below:

quickget output

```text rowdy@silicon:quickemu$ ./quickget --list grep: invalid option -- P usage: grep [-abcdDEFGHhIiJLlMmnOopqRSsUVvwXxZz] [-A num] [-B num] [-C[num]] [-e pattern] [-f file] [--binary-files=value] [--color=when] [--context[=num]] [--directories=action] [--label] [--line-buffered] [--null] [pattern] [file ...] grep: invalid option -- P usage: grep [-abcdDEFGHhIiJLlMmnOopqRSsUVvwXxZz] [-A num] [-B num] [-C[num]] [-e pattern] [-f file] [--binary-files=value] [--color=when] [--context[=num]] [--directories=action] [--label] [--line-buffered] [--null] [pattern] [file ...] OS Release Option alma 8 boot alma 8 dvd alma 8 minimal alma 9 boot alma 9 dvd alma 9 minimal alpine v3.11 alpine v3.12 sparkylinux /> content="7 sparkylinux /> content="7 sparkylinux /> content="7 sparkylinux /> content="7 sparkylinux /> kde sparkylinux /> kde sparkylinux /> lxqt sparkylinux /> lxqt sparkylinux /> mate sparkylinux /> mate sparkylinux /> minimalcli sparkylinux /> minimalcli sparkylinux /> minimalgui sparkylinux /> minimalgui sparkylinux /> name="twitter:description" sparkylinux /> name="twitter:description" sparkylinux /> property="og:description" sparkylinux /> property="og:description" ``` The rest of the list, from `spirallinux` (which appears after `sparkylinux`) onwards, appears to be valid. There are around 1536 lines of `sparkylinux` nonsense, similar to the above.

System information Run quickreport and include the output here; if you can't run quickreport, please provide the output of the following:

Quickreport output

```text ---------------------------------- Quickemu 4.9.6 ---------------------------------- Distro: macOS 14.5 Kernel: Darwin 23.5.0 arm64 Memory: 64G GPU: -Apple M2 Max ---------------------------------- curl 8.6.0 ---------------------------------- Libraries: libcurl/8.6.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.61.0 Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets ---------------------------------- QEMU 9.0.2 ---------------------------------- Available CPUs: a64fx arm1026 arm1136 arm1136-r2 arm1176 arm11mpcore arm926 arm946 cortex-a15 cortex-a35 cortex-a53 cortex-a55 cortex-a57 cortex-a7 cortex-a710 cortex-a72 cortex-a76 cortex-a8 cortex-a9 cortex-m0 cortex-m3 cortex-m33 cortex-m4 cortex-m55 cortex-m7 cortex-r5 cortex-r52 cortex-r5f host max neoverse-n1 neoverse-n2 neoverse-v1 pxa250 pxa255 pxa260 pxa261 pxa262 pxa270-a0 pxa270-a1 pxa270 pxa270-b0 pxa270-b1 pxa270-c0 pxa270-c5 sa1100 sa1110 ti925t ---------------------------------- CPU ---------------------------------- Apple M2 Max ```

Screenshots n/a

Additional context n/a

[lj3954]

Quickget uses pattern matching to find operating system releases, editions, and URLs, when such data can't be accessed via APIs. SparkyLinux appears to have modified their download page since quickget's release/edition detection for it was last updated.

[flexiondotorg]

Sparky Linux listings are changing frequently. I am inclined to unlist Sparky.

[lj3954]

Sparky Linux listings are changing frequently. I am inclined to unlist Sparky.

You're trying to parse data from a user facing page, one which is likely to change quite often. The direct mirror would likely be a better option. http://us2.repo.sparkylinux.org/iso/stable/

[philclifford]

1406 should take care of Sparky errors so that just leaves the grep -- invalid option

[lj3954]

macOS ships BSD grep, which doesn't support perl expressions. All instances of grep -P must be replaced with grep -E to fix this.

[philclifford]

$ grep -n -e grep ../quickget | grep -e '-P'
845:    echo unstable $(web_pipe "https://nix-channels.s3.amazonaws.com/?delimiter=/" | grep -o -P '(?<=<Key>nixos-)[0-9]+.[0-9]+(?=</Key>)' | sort -nr | head -n +2)
882:    echo $(web_pipe "https://download.parrot.sh/parrot/iso/" | grep -o -P '(?<=href=")[0-9].*(?=/")' | sort -nr | head -n 1)

[rowdy-fms]

I've tried simply replacing -P with -E (in both places), but that resulted in the following error appearing twice:

grep: repetition-operator operand invalid

My regex-fu is not sufficiently developed to resolve that error.

[lj3954]

I've tried simply replacing -P with -E (in both places), but that resulted in the following error appearing twice:

grep: repetition-operator operand invalid

My regex-fu is not sufficiently developed to resolve that error.

You can't simply replace the flag, the regex syntax is different.

[philclifford]

@rowdy-fms These tweaks could doubtless be improved, but they work and are fairly readable:

diff --git a/quickget b/quickget
index ab9cd3c174..4118932b98 100755
--- a/quickget
+++ b/quickget
@@ -842,7 +842,7 @@ function releases_nitrux() {
 function releases_nixos() {
     # Lists unstable plus the two most recent releases
     #shellcheck disable=SC2046
-    echo unstable $(web_pipe "https://nix-channels.s3.amazonaws.com/?delimiter=/" | grep -o -P '(?<=<Key>nixos-)[0-9]+.[0-9]+(?=</Key>)' | sort -nr | head -n +2)
+    echo unstable $(web_pipe "https://nix-channels.s3.amazonaws.com/?delimiter=/" | grep -o -E 'nixos-[[:digit:]]+\.[[:digit:]]+' | cut -d- -f2 | sort -nru | head -n +2)
 }

 function editions_nixos() {
@@ -879,7 +879,7 @@ function releases_oraclelinux() {

 function releases_parrotsec() {
     #shellcheck disable=SC2046,SC2005
-    echo $(web_pipe "https://download.parrot.sh/parrot/iso/" | grep -o -P '(?<=href=")[0-9].*(?=/")' | sort -nr | head -n 1)
+    echo $(web_pipe "https://download.parrot.sh/parrot/iso/" |  grep -o -E 'href="[[:digit:]]\.[[:digit:]]+' | sort -nr | head -n 3 | cut -d\" -f 2 )
 }

 function editions_parrotsec() {