k1LoW / runn

runn is a package/tool for running operations following a scenario.
MIT License
393 stars 30 forks source link

Add before/after Hooks for Efficient Testing #612

Open jnst opened 9 months ago

jnst commented 9 months ago

Currently, I am utilizing runn with YAML to execute API tests, which includes the creation and cleanup of test data. The specific method of description is as follows:

steps:
  clean:
    db:
      query: |
        DELETE FROM
          product
        WHERE
          product_id = '{{ vars.product_id }}'
  create:
    db:
      query: |
        INSERT INTO product (
          product_id,
          name
        ) VALUES (
          '{{ vars.product_id }}',
          '{{ vars.name }}'
        )
  someTest:
    req:
      /products

However, using this method, the data in the table inevitably remains after the test execution. Alternatively, if I move the clean step to the end and a test fails midway and is re-run, a duplicate key error occurs, making management difficult.

To alleviate this, I request the addition of a feature that allows the use of hooks commonly used in unit testing, such as before/after and beforeAll/afterAll.

before:
  create:
    db:
      query: |
        INSERT INTO product (
          product_id,
          name
        ) VALUES (
          '{{ vars.product_id }}',
          '{{ vars.name }}'
        )
after:
  clean:
    db:
      query: |
        DELETE FROM
          product
        WHERE
          product_id = '{{ vars.product_id }}'
steps:
  someTest:
    req:
      /products

This will likely enable more effective and efficient preparation and cleanup of test data.

k1LoW commented 9 months ago

@jnst Thank you for your idea.

It's a similar issue to https://github.com/k1LoW/runn/issues/397.

When runn is used as a Go package, BeforeFunc and AfterFunc are available.

We are cautious about providing before/after (or fail) in YAML for this issue because it does not offer the same flexibility as Go.

With the current functionality, something similar could be achieved using the force: section.

In any case, we'll consider it carefully.