DevExpress / testcafe-browser-provider-saucelabs

This is the Sauce Labs browser provider plugin for TestCafe.
https://devexpress.github.io/testcafe/
MIT License
31 stars 41 forks source link

Imported Role instances not working as intended. #55

Closed Joao-S-Martins closed 4 years ago

Joao-S-Martins commented 4 years ago

My code works perfectly in all local browsers but fails consistently with Sauce Labs remote browsers. I create an instance of a Role in a module and then import it to be a parameter in a useRole call. When run locally, everything functions correctly. When run remotely, it walks through the login process but gets kicked back to the login screen as though the session data hasn't been retained.

Module

let admin, instance;

class Helper {
  constructor() {
    if (!instance) {
      instance = this;
    }
    return instance;
  }

  get admin() {
    if (!admin) {
      admin = roleFactory('https://www.example.com', {email: 'some@thing.com', password: 'abc123'});
    }
    return admin;
  }

  roleFactory(url, def) {
    let role = Role(url, async t => {
      // Interact with page
    }, { preserveUrl: true });
    return Object.assign(role, {def});
  }
}

export default Helper;

Test file snippet

const helper = new Helper();
fixture `smoke tests`
  .page(url)
  .beforeEach(async () => await t.useRole(helper.admin));

In my investigation, I moved pieces of the Helper code into the test code file and everything worked, such as

  .beforeEach(async () => await t.useRole(helper.roleFactory(url, helper.admin.def)));

In this case, helper.admin is still the same role instance with a def attribute attached to it as before. Somehow though, passing helper.admin to useRole impacts test runs on Sauce Labs in ways that it doesn't for local runs.

alexey-lin commented 4 years ago

Hi @Joao-S-Martins,

Could you provide us with a sample to reproduce the problem on our side? I looked at the snippets, but have no idea what might go wrong.

Did you try to move the Helper class code to the test code file entirely? Does it help?

  .beforeEach(async () => await t.useRole(helper.roleFactory(url, helper.admin.def)));

I believe, there's an error here: roleFactory itself doesn't cache the instance (helper.admin does), so a new role is used every time.

Joao-S-Martins commented 4 years ago

@alexey-lin Thanks for the response. I just tried copying the class into the test file and the same issues occur, where it works perfectly locally but won't log in with Sauce Labs.

Thanks for the details about the role factory. I did some more testing around that aspect and noticed that setting the return value of roleFactory() to a variable from outside of a test, then passing that to useRole() within a test, also fails on Sauce Labs though is works locally. But setting that variable within the test and then using it in the same test works in both environments. Same deal within a beforeEach() as well.

I forgot to mention what the erroneous run looks like on Sauce Labs: The initial URL loads and reloads three times (inline with use the -q flag), but after that, it reloads continuously with the hash part of the URL changing. I think this hash bit is actually due to my login flow though. The point is that upon loading the page URL, none of the Role functionality seems to take place.

Joao-S-Martins commented 4 years ago

I've done some more experimenting and this may actually be an issue rooted within my authentication. I'm able to get the right code to work on Sauce Labs when only running a single browser. I'm able to reproduce the issue in my local dev under some specific circumstances as well. So this might have more to do with the way my auth code works during multi-browser runs. Thanks.