Test Bash Script returning Exit 1 for passing tests

[alicenwonderland94]

Description of the bug/issue

When I run a bash script command to run nightwatch tests, I expect the script to return an EXIT code of 0 when all tests are passing but instead I'm getting an EXIT code of 1. The console output shows no errors.

Nightwatch.js Version

2.6.13

Node Version

16.13.1

[gravityvi]

@alicenwonderland94 just to understand the use case a bit better what does the bash script do?

[alicenwonderland94]

@alicenwonderland94 just to understand the use case a bit better what does the bash script do?

@gravityvi This script helps us to run our test suite on Jenkins, setting the devices to be used and which one of our marketplace websites it needs to run on.

[gravityvi]

Can we verify if the Nightwatch process returns a non-zero exit code?

[alicenwonderland94]

@gravityvi Yes, the Nightwatch process is returning an exit code of 1.

[gravityvi]

Could you create an example to recreate this? It would help us debug things faster.

[alicenwonderland94]

@gravityvi Hi, I'm not able to reproduce this consistently. Is there some kind of logging that I can possibly provide?

[gravityvi]

@alicenwonderland94 Yes that might help, also if you can provide a snippet of the bash script that would help us verify the issue.

[alicenwonderland94]

@gravityvi What kind of logging would you prefer? Our regular console logs don't provide much info other that what's shown in the image provided above. Here's a snippet of our code and the command we're running

#!/usr/bin/env bash

# Setup defaults
ENV=${ENV-"alpha"}

# Sets the marketplace abbreviation for the device config the test command will execute
case ${MARTY_MARKETPLACE} in
    6pm)
        MKTP=6
        ;;
    zappos)
        MKTP=zap
        ;;
esac

# Sets the device config that the test command will execute
case ${DEVICE} in
    desktop)
        DEVICE=${MKTP}_${ENV}_bs.win11_chrome_desktop
        ;;
    tablet)
        DEVICE=${MKTP}_${ENV}_bs.win11_chrome_tablet
        ;;
    mobile)
        DEVICE=${MKTP}_${ENV}_bs.android_pixel7
        ;;
    other)
        DEVICE=${MKTP}_${ENV}_${OTHER_DEVICE}
esac

date
cd packages/e2e
npm ci
npx patch-package
date

set +e

npx nightwatch -e ${DEVICE} --parallel $1 $2 $3 $4 $5 $6 $7 $8 $9

EXIT_CODE=$?

date
echo "Test run exit code: $EXIT_CODE"

exit ${EXIT_CODE}

MARTY_MARKETPLACE=6pm \
ENV=alpha \
DEVICE=other \
OTHER_DEVICE=bs.galaxy_s21 \
bash -ex ./scripts/nw2_jenkins_e2e/nw2_jenkins.sh 10 --suiteRetries 2