uutils / coreutils

Cross-platform Rust rewrite of the GNU coreutils
https://uutils.github.io/
MIT License
17.23k stars 1.24k forks source link

CI: UTILITY_LIST list of tools is empty, used for cargo clippy in CICD/FreeBSD workflows #6479

Closed lcheylus closed 1 week ago

lcheylus commented 2 weeks ago

In CI, the UTILITY_LIST variable is used by cargo clippy to check lint errors (list of tools to test according to the selected features):

This list is generated by the ./util/show-utils.sh script depending on features used in workflow.

With my previous fix (commit https://github.com/uutils/coreutils/commit/3b96ff1d10a4d74e7018c6a84474de75d168136c), this list is correctly generated by ./util/show-utils.sh script for the different versions of Rust/cargo:

$ ./util/show-utils.sh --features unix
arch base32 base64 basename basenc cat chgrp chmod chown chroot cksum comm cp csplit cut date dd df dir dircolors dirname du echo env expand expr factor false fmt fold groups hashsum head hostid hostname id install join kill link ln logname ls mkdir mkfifo mknod mktemp more mv nice nl nohup nproc numfmt od paste pathchk pinky pr printenv printf ptx pwd readlink realpath rm rmdir seq shred shuf sleep sort split stat stdbuf stty sum sync tac tail tee test timeout touch tr true truncate tsort tty uname unexpand uniq unlink uptime users vdir wc who whoami yes

But in some runs of CICD and FreeBSD workflows, the UTILITY_LIST variable is empty:

I don't find why the UTILITY_LIST variable is OK when using the ./util/show-utils.sh script in command-line (tested on Linux/Debain and OpenBSD) but NOK when used in GitHub workflows.

lcheylus commented 2 weeks ago

Due to this issue, some lint errors are not detected by cargo clippy in CICD and FreeBSD workflow: sources for some tools are not checked => not included in UTILITY_LIST list by default.

lcheylus commented 1 week ago

After some debug/tests, I found the root cause of this issue: there is an mistake in the jq request used in util/show-utils.sh script used by CICD and FreeBSD workflow => due to my last commit 3b96ff1d10a4d74e7018c6a84474de75d168136c

$ cargo metadata --format-version 1|jq -r '[.resolve.nodes[] | select(.id|match(".*coreutils")) ]'|head -3 [ { "id": "path+file:///home/fox/dev/rust-coreutils.git#coreutils@0.0.26",

=> **the jq request with regexp is OK and returns the correct list of tools**.

The regexp for `jq` must be fixed to support each case (local path = / != `<...>/coreutils`) => `match(".*coreutils[ |@|#]\\d+\\.\\d+\\.\\d+"))`

With the regexp version and local path = `<...>/coreutils/`:
```bash
$ cargo metadata --format-version 1|jq -r '[.resolve.nodes[] | select(.id|match(".*coreutils[ |@|#]\\d+\\.\\d+\\.\\d+"))]'|head -3
[
  {
    "id": "path+file:///home/fox/dev/coreutils#0.0.26",

$ cargo metadata --format-version 1 | jq -r '[.resolve.nodes[] | select(.id|match(".*coreutils[ |@|#]\\d+\\.\\d+\\.\\d+")) | .deps[] | select(.pkg|match("uu_")) | .name | sub("^uu_"; "")] | sort | join(" ")'
base32 base64 basename basenc cat cksum comm cp csplit cut date dd df dir dircolors dirname du echo env expand expr factor false fmt fold hashsum head join link ln ls mkdir mktemp more mv nl numfmt od paste pr printenv printf ptx pwd readlink realpath rm rmdir seq shred shuf sleep sort split sum tac tail tee test touch tr true truncate tsort unexpand uniq unlink vdir wc yes

$ cargo metadata --features unix --format-version 1 | jq -r '[.resolve.nodes[] | select(.id|match(".*coreutils[ |@|#]\\d+\\.\\d+\\.\\d+")) | .deps[] | select(.pkg|match("uu_")) | .name | sub("^uu_"; "")] | sort | join(" ")'
arch base32 base64 basename basenc cat chgrp chmod chown chroot cksum comm cp csplit cut date dd df dir dircolors dirname du echo env expand expr factor false fmt fold groups hashsum head hostid hostname id install join kill link ln logname ls mkdir mkfifo mknod mktemp more mv nice nl nohup nproc numfmt od paste pathchk pinky pr printenv printf ptx pwd readlink realpath rm rmdir seq shred shuf sleep sort split stat stdbuf stty sum sync tac tail tee test timeout touch tr true truncate tsort tty uname unexpand uniq unlink uptime users vdir wc who whoami yes