SPIKE: Determine if AVA test framework can be used with SES

david0xd commented 1 year ago

Determine if AVA test framework can be used to test code which includes SES functions.

It is found that the Jest test framework does not work with SES. That makes impossible to test code that includes SES functions such as harden({...}).

Given that the Jest is used in the snaps-execution-environments package it is not possible to continue with a ticket for hardening all endowments, so the research is needed in order to replace the test framework in the package mentioned.

david0xd commented 1 year ago

After some research it is confirmed that SES can be used with the AVA test runner.

SES notes:

There are some lockdown specific configurations that needs to be applied like the one below:

lockdown({ domainTaming: 'unsafe', errorTaming: 'unsafe' });

Otherwise, SES would be throwing an error related to the domains:

  Error thrown in test:

  TypeError {
    message: 'SES failed to lockdown, Node.js domains have been initialized (SES_NO_DOMAINS)',
  }

Given that the unsafe options are only used in tests it should not be a major concern for now.

Testing code that use SES functions:

In order to test code that is using SES functions, such as harden(), test conversion should be made. Conversion means that, at least, all the tests that are using some of the SES functionality must be converted to be run by AVA test runner (e.g. all endowment tests).

AVA notes:

Because of the different nature of AVA test runner, some features available in Jest will not be available in AVA. Jest is using Jasmine approach for describing and grouping tests, in AVA it is not available in that way. Jest natively supports function mocking (out of the box), AVA does not. That means the external library for that purpose will be needed (recommended: https://sinonjs.org).

Example and comparison of the tests in Jest and AVA respectively: Jest:

it('teardownFunction should clear timeouts', async () => {
    const { setTimeout: _setTimeout, teardownFunction } = timeout.factory();

    expect(
      await new Promise((resolve, reject) => {
        _setTimeout(reject, 100);
        teardownFunction();
        setTimeout(resolve, 200);
      }),
    ).toBeUndefined();
  }, 300);

AVA:

test('teardownFunction should clear timeouts', async (t) => {
  const { setTimeout: _setTimeout, teardownFunction } = timeout.factory();

  const result = await new Promise((resolve, reject) => {
    _setTimeout(reject, 100);
    teardownFunction();
    setTimeout(resolve, 200);
  });

  t.is(result, undefined);
});

Code example of importing SES and configuring lockdown with AVA:

// eslint-disable-next-line import/no-unassigned-import
import 'ses';
import test from 'ava';

test.before(() => {
  lockdown({ domainTaming: 'unsafe', errorTaming: 'unsafe' });
});

Other notes

Test conversion strategy is to be determined. Research PR with examples is available here: https://github.com/MetaMask/snaps-monorepo/pull/1008/files

david0xd commented 1 year ago

Since the research on the topic was completed and there are no additional inputs from the team, I'm closing this ticket. The related work will be followed within the epic: https://github.com/MetaMask/snaps-monorepo/issues/585

MetaMask / snaps