pluto-lang / pluto

Pluto provides a unified programming interface that allows you to seamlessly tap into cloud capabilities and develop your cloud and AI applications.
https://pluto-lang.vercel.app
Apache License 2.0
86 stars 8 forks source link

Support unit testing feature #42

Open jianzs opened 11 months ago

jianzs commented 11 months ago

Enable the unit testing feature, allowing users to select either a simulator or real runtime as their testing environment. Testing in the real runtime helps prevent issues where code functions correctly locally but fails in the online environment. Testing in the simulator is faster and allows for quick identification of problems.

The current idea is to provide users with a user interface similar to vitest. During testing, each unit test will be given an independent environment to avoid interference between tests. At the same time, only the resources related to each unit test will be deployed, in order to avoid resource waste and save time.

To support testing, add a resource called "Tester". The implementation for Tester will output the ID of all function resources. This ID will be used to invoke the test function.

import { Tester } from "@plutolang/pluto";
import assert from "assert";

const tester = new Tester("function call");

function add(a: number, b: number): number {
  return a + b;
}

tester.it("add case1", async () => {
  assert(add(1, 3) == 3);
});

test.it("add case2", async () => {
  assert(add(1, 3) == 4);
});

Deducer will analyze the code as usual. The CLI will divide the architecture reference into multiple groups based on testers. Each group represents a new architecture reference and includes only one tester along with the necessary resources for that tester. Each group will be tested separately.

The CLI will deploy each architecture reference, retrieve the function resource ID from the adapter's outputs, and create a TesterClient to invoke each test function.

jianzs commented 10 months ago

The current setup puts the TesterClient in the CLI, which limits its flexibility. This means that if users want to use their own SDK to create a new Tester, they have to add the Client implementation in the CLI first. In order to improve this, we should consider moving the TesterClient into the SDK. However, we also need a way for the CLI to determine which SDK's TesterClient should be used.