xhd2015 / xgo

All-in-one go testing library
MIT License
368 stars 19 forks source link

Discussion: data driven testing #52

Open xhd2015 opened 7 months ago

xhd2015 commented 7 months ago

Never depend on the implementation directly. Abstract a layer between the implementation and the intent.

For example, when testing a service with external dependencies, we want to mock all of them.

But we should not directly mock their request or response, instead, we abstract a data structure called TestingContext.

This TestingContext is declared as a universal data fake storage:

type TestingContext struct {
    Users []*User
    Checkouts []*Checkout
    Config *Config
}

Then each dependency read data from this central data storage, and construct corresponding response based on the data.

And to reduce duplication of repeating declaring TestingContext, different tests can share a basic TestingContext, adding their custom overlay to the context.

xhd2015 commented 7 months ago

Two important concepts:

xhd2015 commented 7 months ago

Data driven requires the user to have a basic knowledge of the system's models as well as APIs to interact with these models.

xhd2015 commented 7 months ago

But data drive testing is already taken, also known as table driven testing. Wiki: https://en.m.wikipedia.org/wiki/Data-driven_testing.

However, given that table driven is widely used and has gained a relative stable shape(a table), then this method can still be referred as Data driven testing.

xhd2015 commented 7 months ago

Data driven requires the user to have a basic knowledge of the system's models as well as APIs to interact with these models.

To do data driven testing, following these 4 steps:

(draw a diagram to illustrate the relation of data and APIs, APIs are fall into 2 categories: internal and external like database.)

xhd2015 commented 7 months ago

Data driven requires the user to have a basic knowledge of the system's models as well as APIs to interact with these models.

To do data driven testing, following these 4 steps:

  • recognize entities in the system,
  • define their ids and properties ,
  • mock the API behavior based on the mocking data

    • this can be done by comparing the querying key with the data context, and retrieve matching result from the context. panic out if no data found
  • construct base data context, and create overlays to repeat the test for different logic paths

(draw a diagram to illustrate the relation of data and APIs, APIs are fall into 2 categories: internal and external like database.)

The reverse process, convert real API response to data context, is also valuable. Compared to that directly replay real traffic, we actually split that into 2 phase:

With this, unit test can be more easily constructed and maintained from live traffic.