[Maestro Cloud] Proposal: Email Validation

[igorsmotto]

Overview

This issue proposes a solution for testing end-to-end flows that include validating that emails are sent, and most importantly, extracting values out of the message to use elsewhere in the test. We would like to hear your feedback.

Background

By simulating user interaction Maestro tries to replicate exactly how a user would normally use your App during a Flow execution.

There are, however, scenarios like new user registration, password reset, user invite, and purchasing, that require that the user interacts with an email sent to the user's email inbox. We consider these advanced use cases.

One known workaround is to create a special user with relaxed security measures so that every action doesn't require leaving the App during test execution. Another method is to attempt to access the email inbox through the device's browser. Both have drawbacks: the former diminishes Maestro's coverage capacity, and the latter adds additional unwanted complexity, increasing flakiness probability.

Proposal

To address this issue, we propose adding a way to both receive and search through emails in a Virtual Mailbox for Maestro Cloud. This will allow users to access their real emails during a flow execution, and most importantly, their content.

Since this problem is considered an advanced use case, this proposal leverages Maestro's Javascript API, adding a new auto-imported function to the Javascript Sandbox:

function findEmail(email: Email, timeout?: number) => Email[];

interface Email {
    to?: string,
    from?: string,
    body?: string,
    subject?: string,
    date?: string
};

This function will poll the Mailbox's API querying for no longer than the timeout duration, as soon it finds one or more emails that match all arguments, it will return back their contents to be used.

At first, this proposal is only for users registered on Maestro Cloud, this is because there is a need to receive real emails from users and securely expose their contents for local executions of Maestro. This is done by using a custom domain email inbox.mobile.dev

Usage

In this usage example, Maestro will execute a signup flow with a specific email address, and it will reuse this email address to find out if any email was sent to this address.

# Input email and complete sign-up
- inputText: "igor@inbox.mobile.dev"
- tapOn: "Complete Sign-Up"
# As the sign-up is completed, app's backend will send an email to "igor@inbox.mobile.dev"

# Now, Maestro will search through the mailbox to check if the email has arrived
- runScript: "email.js"

# Use content from email to navigate within the App (deep-link)
- openLink: ${output.acceptURL}

// Search any email that was sent to igor@example.com
const emails = findEmail({
   to: "igor@example.com",
}, 3000);

// Get body of the email
const body = emails[0].body;

// Add to the output object the URL from the email's content 
const urlRegex = /(https?:\/\/[^\s]+)/g;
output.acceptURL = body.match(urlRegex)[0];

[igorsmotto]

SMS

As you may know, SMS is also used for Mobile applications for these exact same problem. However, we've not ye figured a clear way to solve this. Let us know if this is important for you.

[dwxw]

Hi @igorsmotto, I'm really excited about Maestro and had identified new user verification as being a blocker to using the product. Our product uses both email and SMS MFA as part of signing up a new user. We currently get Appium to call the AWS SDK to manage test users. I was actually wanting to ask if there is way to call their SDK, as it doesn't appear possible. Having a simple solution to this problem in Maestro would be absolutely awesome. The suggested email solution is good, although it would be a shame the tests couldn't be run locally (maybe if there was a way for a local test to wait whilst the verification was entered on stdin, that might suffice). SMS is harder, I sometimes use on of the free online SMS numbers to capture the OTP, maybe there is a service with an API that could be used by Maestro.

[Hosam-hsm]

Any update on this?