[Feature Request] Allow for easy mocking

[lorensr]

Is your feature request related to a problem? Please describe.

Mocking an SDK function is difficult. If you use a system like Sinon that depends on default exports, it doesn't work, since we don't do default exports. Even if you use a system like Jest that allows you to mock the whole package, there are two issues:

• Unless your code only uses a single function, you have to mock or add back in the other functions. This, for instance, creates a package that only exports proxyActivities:

const mockRequest = jest.fn<Promise<string>, []>();
mockRequest.mockResolvedValue('3');

test('httpWorkflow with mocked proxyActivities', async () => {
  jest.mock('@temporalio/workflow', () => ({
    proxyActivities: mockRequest,
  }));

  expect(await httpWorkflow()).toBe('The answer is 3');
});

• When you mock, you have to recreate the environment. For instance, if you run the above, you get:

 FAIL  src/workflows.test.ts (9.489 s)
  ● httpWorkflow with mocked proxyActivities

    TypeError: exports.storage.getStore is not a function

      14 |
      15 | export async function httpWorkflow(): Promise<string> {
    > 16 |   const answer = await makeHTTPRequest();
         |                        ^
      17 |   return `The answer is ${answer}`;
      18 | }
      19 |

      at Function.current (node_modules/@temporalio/workflow/src/cancellation-scope.ts:156:20)
      at node_modules/@temporalio/workflow/src/workflow.ts:151:37
      at scheduleActivityNextHandler (node_modules/@temporalio/workflow/src/workflow.ts:150:10)
      at scheduleActivity (node_modules/@temporalio/workflow/src/workflow.ts:259:10)
      at node_modules/@temporalio/workflow/src/workflow.ts:493:18
      at httpWorkflow (src/workflows.ts:16:24)
      at Object.<anonymous> (src/workflows.test.ts:67:28)

Solution

The current solution in the case of proxyActivities is to run a Worker with a test connection and mock activities (like this), and to start a workflow using the test client. If you want to test a helper function used by workflows, then tell the Worker the helper is a workflow and start it, like this:

// workflow-helpers.ts
export function myHelper() { ... }

// workflow-helpers.test.ts
  const worker = await Worker.create({
    workflowsPath: require.resolve('./workflow-helpers'),
    ...
  });
  await withWorker(worker, async () => {
    const result = await workflowClient.execute(myHelper, {
      workflowId: uuid4(),
      taskQueue: 'test',
    });
    expect(result).toEqual('The answer is 99');
  });  

Feedback

If the current solution described above doesn't work for your case, or isn't ideal for you, let us know in this thread! ☺️

[bergundy]

You can mock the workflow functions but you need to it from the vm context otherwise it’ll have no effect. I agree it’s hard to set up but probably possible with sinon (haven’t tried though). Not sure how much value there is to mocking parts of the workflow instead of using the test environment

[rcarton]

After a lot of trial and error, the least obtrusive method I've found is to use rewire and sinon:

// In workflow, must use `let` instead of const
let { getCustomerActivity } =
  proxyActivities<typeof activities>({
    startToCloseTimeout: "1 minute",
  });

// Test utils
type Activities = typeof activities;
type StubbedActivity<T extends keyof Activities> = sinon.SinonStub<
  Parameters<Activities[T]>,
  ReturnType<Activities[T]>
>;

// Taken from https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/rewire/index.d.ts
interface RewiredModule {
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  __set__(obj: { [variable: string]: any }): () => void;

  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  __set__(name: string, value: any): () => void;

  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  __get__<T = any>(name: string): T;

  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  __with__(obj: { [variable: string]: any }): (callback: () => any) => any;
}

/**
 * Stub an activity
 *
 * @param sandbox a sandbox that you will be restore()'d after the test
 * @param module a rewired module
 * @param activity the name of the activity you want to stub
 */
export function getProxyActivityStub<T extends keyof Activities>(
  sandbox: sinon.SinonSandbox,
  module: RewiredModule,
  activity: T
): StubbedActivity<T> {
  const stub = sandbox.stub();
  module.__set__(activity, stub);
  return stub as StubbedActivity<T>;
}

// In test file
const Workflow = rewire(
  "@temporal-worker/workflows/workflow"
);

const sandbox = sinon.createSandbox();

const getCustomerActivityStub = getProxyActivityStub(
  sandbox,
  Workflow,
  "getCustomerActivity"
);

[bergundy]

Glad you got it working, we'll be working on improving this experience in the future

[mjameswh]

For the record and public visibility, here is a working proof of concept I developed a few months ago for mocking child workflows. The great things with this approach are 1) multiple tests can be performed using the same workflow.ts file (other methods require one workflow.ts per test case), 2) it ends up calling a callback function in the unit test context, so its easy to make test assertions there. This is probably not the way we'll end up implementing this in SDK, but can be useful to users that want this now. See below for limitations.

It works in three steps: first, an interceptor is used to catch startChildWorkflow calls from the Workflow under test, and replaces the workflow type of the child workflow to be started by a predefined "mock workflow" type. Second, the implementation of that "mock workflow" type calls an activity named "mockWorkflowActivity", passing it inputs received from the Workflow under test. Finally, activity type "mockWorkflowActivity" is registered directly on the test Worker.

Are are implementation instructions:

1. In workflow.ts (the one used by unit test), add this workflow:

   export async function mockChildWorkflow(...args: unknown[]): Promise<string> {
   return await scheduleActivity('mockWorkflowActivity', args, { scheduleToCloseTimeout: '5s' });
   }

2. Next to test files, create a file named mock-child-workflow-interceptor.ts, containing the following code:

   
   import { WorkflowInterceptorsFactory } from '@temporalio/workflow';

export const interceptors: WorkflowInterceptorsFactory = () => { return { outbound: [ { startChildWorkflowExecution(input, next) { return next({ ...input, options: { ...input.options, args: [input.workflowType, ...input.options.args], }, workflowType: 'mockChildWorkflow', }); }, }, ], }; };

3. In unit test where we want to mock child workflows, define a function named `mockWorkflowActivity` and register it as an activity on the worker; also register the previously created interceptor. Assuming mocha, that could for example give:

it('Mock child workflows', async () => { const { client, nativeConnection } = testEnv;

function mockWorkflowActivity(workflowType: string, ...args: unknown[]): unknown {
  return 'test';
}

const worker = await Worker.create({
  connection: nativeConnection,
  taskQueue: 'test',
  workflowsPath: require.resolve('./workflows'),
  activities: { ...activities, mockWorkflowActivity },
  interceptors: { workflowModules: [require.resolve('./mock-child-workflow-interceptor')] },
});

await worker.runUntil(async () => {
  const result = await client.workflow.execute(parentWorkflow, {
    workflowId: uuid4(),
    taskQueue: 'test',
    args: ['alice', 'bob', 'charlie'],
  });
  assert.equal(result, 'test\ntest\ntest');
});

}).timeout(20000);

Note some caveats with this implementation:
- Exceptions — Throwing from the mock workflow activity won’t result in the same thrown exception at the workflow level.
- Timing — The mock workflow activity will have to return pretty quickly, where as the actual child workflow may have stayed alive for some time.
- Signals — This implementation won’t work if you need the parent and child workflows to exchange signals.
- All or nothing — The current implementation intercepts all child workflow executions. It can’t be selective on which invocations are to be catch. It would not be hard however to fix.

[bergundy]

I like where you're going with this @mjameswh.

A few places where this can be expanded:

• Integration of this solution and make it easier to use and more configurable
• Integrations with mock frameworks as config packages
• Give control over which workflow types get mocked
• Make this not an activity so it (possibly, need to verify) doesn't affect time skipping (would need to introduce a new concept (*))
• Try an approach where mocks run in workflow context
• Add signal support
• Add exception translation support

* We were talking about a local activity that doesn't produce a marker that can be used in query handlers and update validators, this could be helpful here too.

[gaurav1999]

Here's an example if anyone's looking to test parent workflows executing child workflows.

Your workflow code -> src/workflows/order.workflow.ts [Parent]

Order workflow code executes a bunch of activities, and one child workflow called sendNotificationWorkflow.

In order to test order.workflow.ts and use mock child workflow sendNotificationWorkflow while testing.

Create a separate file src/workflow/tests/test-order.workflow.ts

export * from '../order.workflow.ts'; //Exporting the original order workflow since we want to test it's original code as it is

//Mock child workflow
export async function sendNotificationWorkflow(
    email: string,

  ): Promise<boolean> {
    return true;
}

And now, put this file in your worker worfklowsPath option so that it loads the code of your original workflow, and mock code of your child workflow

const worker = await Worker.create({
      connection: nativeConnection,
      taskQueue,
      workflowsPath: require.resolve('./test-order.workflow.ts'),
      activities,
      debugMode: true
 });

This works well for me!

I think, the idea is, that it's easier to give a mocked context to worker, rather rewiring the SDK components.