TestArmada / magellan

Large Scale Automated Testing for Mocha, Nightwatch, Appium, Nodejs, etc
MIT License
287 stars 44 forks source link

Profiles in Magellan.json reverting to Default Browser Settings #236

Closed joewdriver closed 7 years ago

joewdriver commented 7 years ago

Hello,

Given the following config:

"profiles": { "tier-one-browsers": [ { "browser": "chrome_latest_OS_X_10_11_Desktop", "resolution": "1280x1024", "executor": "seleniumgrid" }, { "browser": "firefox_latest_OS_X_10_11_Desktop", "resolution": "1280x1024", "executor": "seleniumgrid" } ] },

and the command:

./node_modules/.bin/magellan --profile tier-one-browsers --test tests/Account/accountSignInGmail.js --debug

I have been trying to run on multiple browsers through our selenium grid, but have been unsuccessful. The grid config is working, because it hits the hub and distributes the tests correctly. It is also correctly detecting the number of profiles, since the number of tests grows with each addition to the profile list.

However, there is no recognition of the different browsers attached here, every test reverts to the firefox default. I've tried mirroring the nightwatch boilerplate as closely as my system allows and have had no luck.

At this point I've been trying to dig around in the node module to figure out if: a. I'm missing some sort of config value b. there is a bug preventing this from working under a set of circumstances.

I am able to run multiple browsers via the command line. From the output of the failed attempts you can see the env is not being set (this value is set when we read from the command line):

[INFO] [Magellan] Running 2 tests with 3 workers with [env:undefined|executor:seleniumgrid, env:undefined|executor:seleniumgrid] [INFO] [Magellan] --> Worker 2, port range: [12003, 12005], running test: tests/Account/accountSignInGmail.js @env:undefined|executor:seleniumgrid [INFO] [Magellan] --> Worker 1, port range: [12000, 12002], running test: tests/Account/accountSignInGmail.js @env:undefined|executor:seleniumgrid

Note: it would also be helpful if in the documentation there were a guide to how to define the browsers in the config file. Having to reverse engineer the structure of the string from examples left me in doubt as to whether I was constructing it correctly.

Thanks for the help,

Joe

archlichking commented 7 years ago

can you let me know where you get this profile?

"profiles": {
"tier-one-browsers": [
{
"browser": "chrome_latest_OS_X_10_11_Desktop",
"resolution": "1280x1024",
"executor": "seleniumgrid"
},
{
"browser": "firefox_latest_OS_X_10_11_Desktop",
"resolution": "1280x1024",
"executor": "seleniumgrid"
}
]
},

this is not a valid profile for selenium grid. the browsers in the profile are for saucelabs, and these browsers cannot be picked up by seleniumgrid-executor. the browser you should use for selebiumgrid is the same as your local browser, please take a look at http://testarmada.io/#localBrowserPreresuisites.

magellan executor will convert the profile to a format it can understand. this chrome_latest_OS_X_10_11_Desktop format can only be understood by saucelabs-executor via guacamole.

to make things much easier, you can simply use the browsers defined in your nightwatch.json for seleniumgrid-executor by

--seleniumgrid_browsers chrome,firefox,safari
joewdriver commented 7 years ago

The profile was pulled from the boilerplate project, then modified for my system. It wasn't clear to me that it was meant for saucelabs only.

I tried using "chrome,firefox" in the profile block as well (it already works from the command line) but I got the same results--the environment variable comes up empty and it defaults to firefox.

It sounds like right now there is no way to set browsers in the profile for seleniumgrid -- is that right, or am I missing something?

archlichking commented 7 years ago

You're right. seleniumgrid-executor doesn't support reading profile from magellan.json right now. We'll mark this as a feature request and implement it soon. For now to run massive tests on seleniumgrid you can use --seleniumgrid_browsers chrome,firefox.

archlichking commented 7 years ago

i'm closing this one as magellan-seleniumgrid-executor is the right place to put the request in. https://github.com/TestArmada/magellan-seleniumgrid-executor/issues/3

joewdriver commented 7 years ago

Thanks, appreciate you taking the time to explain!