CI tests verifying bash version support

[sdolenc]

First of all: great work! I especially love the coverage feature.

I put together some circleci tests that runs the test script with different bash versions

• debian: https://github.com/sdolenc/critic.sh/blob/test-bash-verions/.circleci/config.yml
• alpine: https://github.com/sdolenc/critic.sh/blob/test-alpine-bash-versions/.circleci/config.yml

I believe the readme said bash 4.1 and above should work, but I can only get debian versions "buster" (bash 5) and "stretch" (bash 4.4) to pass successfully.

Would you prefer I describe the errors I'm seeing as separate issues or each one within this issue?

[Checksum]

Thanks for catching this. I've actually not tested this in various bash versions, so this is helpful. The features required should be supported in 4.1 according to the bash changelog.

Can you please link/paste the errors here?

[sdolenc]

comment 1/3

debian "buster" (bash 5) and "stretch" (bash 4.4) tests pass with the following message

--- Coverage report
Exit codes match
Coverage report matches
--- _output_contains
Output matches
CircleCI received exit code 0

alpine-based containers with bash 5 and 4.4 display this message

--- Coverage report
awk: bad regex '^ *(function)?.+\(\).*{$': Invalid contents of {}
Exit codes match
--- /dev/fd/63
+++ /dev/fd/62
@@ -1,6 +1,6 @@
 /root/project/examples/lib.sh
 Total LOC: 19
 Covered LOC: 3
-Coverage %: 11
+Coverage %: 50
 Ignored LOC: 5
-Uncovered Lines: 1 2 3 5 6 7 8 9 11 12 18 19 20 21 22 23 24 25 27 28 29 30 31
+Uncovered Lines: 21 22 30
--- _output_contains
Output matches

Exited with code exit status 1
CircleCI received exit code 1

comment 1/3

[sdolenc]

comment 2/3

debian "jessie" (bash 4.3) and "wheezy" (4.2) outputs

--- Coverage report
critic.sh: line 179: _args[@]: unbound variable
critic.sh: line 280: heredocs[@]: unbound variable
critic.sh: line 302: heredocs[@]: unbound variable
Exit codes match
0a1,6
> /root/project/examples/lib.sh
> Total LOC: 19
> Covered LOC: 3
> Coverage %: 50
> Ignored LOC: 5
> Uncovered Lines: 21 22 30
--- _output_contains
Output matches

Exited with code exit status 1
CircleCI received exit code 1

alpine-based containers with bash 4.3 and 4.2 display this message

--- Coverage report
critic.sh: line 179: _args[@]: unbound variable
awk: bad regex '^ *(function)?.+\(\).*{$': Invalid contents of {}
critic.sh: line 288: empty_lines[@]: unbound variable
critic.sh: line 295: empty_lines[@]: unbound variable
Exit codes match
--- /dev/fd/63
+++ /dev/fd/62
@@ -0,0 +1,6 @@
+/root/project/examples/lib.sh
+Total LOC: 19
+Covered LOC: 3
+Coverage %: 50
+Ignored LOC: 5
+Uncovered Lines: 21 22 30
--- _output_contains
Output matches

Exited with code exit status 1
CircleCI received exit code 1

comment 2/3

[sdolenc]

comment 3/3

debian "squeeze" (bash 4.1) output

--- Coverage report
critic.sh: line 179: _args[@]: unbound variable
critic.sh: line 220: declare: -g: invalid option
declare: usage: declare [-aAfFilrtux] [-p] [name[=value] ...]
critic.sh: line 221: declare: -g: invalid option
declare: usage: declare [-aAfFilrtux] [-p] [name[=value] ...]
critic.sh: line 238: bar: unbound variable
Exit codes match
0a1,6
> /root/project/examples/lib.sh
> Total LOC: 19
> Covered LOC: 3
> Coverage %: 50
> Ignored LOC: 5
> Uncovered Lines: 21 22 30
--- _output_contains
Output matches

Exited with code exit status 1
CircleCI received exit code 1

an alpine container with bash 4.1 outputs similar results

--- Coverage report
critic.sh: line 179: _args[@]: unbound variable
critic.sh: line 220: declare: -g: invalid option
declare: usage: declare [-aAfFilrtux] [-p] [name[=value] ...]
critic.sh: line 221: declare: -g: invalid option
declare: usage: declare [-aAfFilrtux] [-p] [name[=value] ...]
critic.sh: line 238: bar: unbound variable
Exit codes match
--- /dev/fd/63
+++ /dev/fd/62
@@ -0,0 +1,6 @@
+/root/project/examples/lib.sh
+Total LOC: 19
+Covered LOC: 3
+Coverage %: 50
+Ignored LOC: 5
+Uncovered Lines: 21 22 30
--- _output_contains
Output matches

Exited with code exit status 1
CircleCI received exit code 1

comment 3/3

[Checksum]

awk: bad regex '^ *(function)?.+\(\).*{$': Invalid contents of {}

This error maybe because Alpine images use awk from Busybox rather than gawk. Alpine containers need this:

https://github.com/Checksum/critic.sh/blob/dcb5de328da8c5b9f9a9e98254c1ea04b1eb33f1/Dockerfile#L9-L10

I'll take a look at the other errors tomorrow.

[sdolenc]

sounds good! the awk/gawk fix you suggested did resolve that issue 👍

https://github.com/sdolenc/critic.sh/commit/b39b4f766de69bb205c71be338d2e32887f9ca2a

[Checksum]

Ok, did some digging into this today. I installed versions 4.1 - 4.3 on my MacBook and all seem to pass fine:

~/projects/critic.sh (git)-[master] % ~/scratchpad/bash/bash-4.1/bash scripts/test.sh
--- Coverage report
Exit codes match
Coverage report matches
--- _output_contains
Output matches

~/projects/critic.sh (git)-[master] % ~/scratchpad/bash/bash-4.3.30/bash scripts/test.sh
--- Coverage report
Exit codes match
Coverage report matches
--- _output_contains
Output matches

~/projects/critic.sh (git)-[master] % ~/scratchpad/bash/bash-4.2.53/bash scripts/test.sh
--- Coverage report
Exit codes match
Coverage report matches
--- _output_contains
Output matches

I have no idea what's causing this 😕 Could you try printing the actual bash version in scripts/test.sh if you get a chance?

[sdolenc]

Hmm. We may be encountering a difference between bash on mac vs. linux. I ran bash --version within and outside of the script and they match. Debian 8, 7, and 6:

• GNU bash, version 4.3.30(1)-release (x86_64-pc-linux-gnu)
• GNU bash, version 4.2.37(1)-release (x86_64-pc-linux-gnu)
• GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu)

I'm able to reproduce the issue locally (screenshots below) I'll try to find time in the next week or two to investigate a cause and hopefully make a fix (unless someone beats me to it).

bash 4.3 on Ubuntu 16

bash 4.2 on Debian 7 Wheezy

If I write a fix then I'll also test on Debian 6 Squeeze which runs bash 4.1 before I submit a PR

[sdolenc]

I submitted a pr https://github.com/Checksum/critic.sh/pull/8 with fixes and automated testing for bash 4.2 and above. I hope that change looks okay for merging.

I haven't enabled tests for bash 4.1 because I'm now seeing this issue

--- Coverage report
critic.sh: line 220: declare: -g: invalid option
declare: usage: declare [-aAfFilrtux] [-p] [name[=value] ...]
critic.sh: line 221: declare: -g: invalid option
declare: usage: declare [-aAfFilrtux] [-p] [name[=value] ...]
critic.sh: line 238: bar: unbound variable
Exit codes match
0a1,6
> /root/project/examples/lib.sh
> Total LOC: 19
> Covered LOC: 3
> Coverage %: 50
> Ignored LOC: 5
> Uncovered Lines: 21 22 30
--- _output_contains
Output matches

Exited with code exit status 1
CircleCI received exit code 1