TestArmada / magellan

Large Scale Automated Testing for Mocha, Nightwatch, Appium, Nodejs, etc
MIT License
287 stars 76 forks source link

CLI bootstrapping appears broken #269

Open swashcap opened 5 years ago

swashcap commented 5 years ago

Problem

Setting up magellan yields a handful of issues. When running magellan --help:

Steps to reproduce

$ mkdir magellan-test
$ cd $_
$ npm init -y
# ...
$ npm i -D testarmada-magellan testarmada-magellan-local-executor
# ...
$ npx magellan --help
[INFO] [Magellan] Loading configuration from default location: ./magellan.json
[INFO] [Magellan] Creating temporary files at: /Users/dudeman/magellan-demo/temp
[INFO] [Magellan] Version:  11.0.3
[INFO] [Magellan] Use --help to list out all command options
[ERROR] [Magellan] Could not start Magellan.
[ERROR] [Magellan] Could not load the testing framework plugin: testarmada-magellan-nightwatch-plugin
[ERROR] [Magellan] Check and make sure your package.json includes module: testarmada-magellan-nightwatch-plugin
[ERROR] [Magellan] Error: Cannot find module '/Users/dudeman/magellan-demo/node_modules/testarmada-magellan-nightwatch-plugin/index'
[ERROR] [Magellan] Error while running Magellan: undefined
[ERROR] [Magellan] undefined
$ echo $?
1
swashcap commented 5 years ago

After adding an empty magellan.json:

$ touch magellan.json
$ npx magellan --help
[INFO] [Magellan] Loading configuration from default location: ./magellan.json
Could not load or parse configuration from /Users/dudeman/magellan-demo/magellan.json

And after adding an empty JSON object:

$ echo '{}' > magellan.json 
$ npx magellan --help
[INFO] [Magellan] Loading configuration from default location: ./magellan.json
Loaded configuration from:  /Users/dudeman/magellan-demo/magellan.json
[INFO] [Magellan] Creating temporary files at: /Users/dudeman/magellan-demo/temp
[INFO] [Magellan] Version:  11.0.3
[INFO] [Magellan] Use --help to list out all command options
[ERROR] [Magellan] Could not start Magellan.
[ERROR] [Magellan] Could not load the testing framework plugin: testarmada-magellan-nightwatch-plugin
[ERROR] [Magellan] Check and make sure your package.json includes module: testarmada-magellan-nightwatch-plugin
[ERROR] [Magellan] Error: Cannot find module '/Users/dudeman/magellan-demo/node_modules/testarmada-magellan-nightwatch-plugin/index'
[ERROR] [Magellan] Error while running Magellan: undefined
[ERROR] [Magellan] undefined
swashcap commented 5 years ago

Does magellan have a "Getting Started" guide? I see a “Test Framework Compatibility and Installation” section in the README, but this has a changelog and explanation of libraries along with install instructions and config. Not exactly approachable.

swashcap commented 5 years ago

I see this page on Testarmada.io, but it doesn't explain how to programmatically set up magellan in a new or existing project.

swashcap commented 5 years ago

Installing the Nightwatch plugin appears to change the error:

$ npm i -D testarmada-magellan-nightwatch-plugin
# ...
$ npx magellan --help
[INFO] [Magellan] Loading configuration from default location: ./magellan.json
Loaded configuration from:  /Users/dudeman/magellan-demo/magellan.json
[INFO] [Magellan] Creating temporary files at: /Users/dudeman/magellan-demo/temp
[INFO] [Magellan] Version:  11.0.3
[INFO] [Magellan] Use --help to list out all command options
[INFO] [Magellan] Loaded test framework from magellan.json: 
[INFO] [Magellan]   testarmada-magellan-nightwatch-plugin
[WARN] [Magellan] No executor is configured
[WARN] [Magellan] Setting executor to [testarmada-magellan-local-executor] by default
[INFO] [Magellan] Loaded test executors from magellan.json: 
[INFO] [Magellan]   testarmada-magellan-local-executor
[ERROR] [Magellan] Error while running Magellan: Cannot find module '/Users/dudeman/magellan-demo/conf/nightwatch.json'
[ERROR] [Magellan] Error: Cannot find module '/Users/dudeman/magellan-demo/conf/nightwatch.json'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:582:15)
    at Function.Module._load (internal/modules/cjs/loader.js:508:25)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at Object.getProfiles (/Users/dudeman/magellan-demo/node_modules/testarmada-magellan-nightwatch-plugin/lib/profile.js:10:30)
    at /Users/dudeman/magellan-demo/node_modules/testarmada-magellan-local-executor/lib/profile.js:55:60
    at new Promise (<anonymous>)
    at Object.getProfiles (/Users/dudeman/magellan-demo/node_modules/testarmada-magellan-local-executor/lib/profile.js:42:12)
    at _.forEach (/Users/dudeman/magellan-demo/node_modules/testarmada-magellan/src/profiles.js:143:48)
    at /Users/dudeman/magellan-demo/node_modules/lodash/lodash.js:4911:15

Is the magellan CLI tightly coupled to nightwatch-extra and the local executor?