One part of the larger testing strategy for ICU4X would be to have a clean, consistent way of organizing the unit tests for the business logic (i18n algorithms). In particular, it would be nice to have a data-oriented style of testing, as exemplified already in some parts of @zbraniecki's unic-locale repo. ICU unit tests tend to be written in a parameterized style, but the idea here is to take the data-driven nature further.
Pros and Cons
Pros:
ICU algorithms are mostly (all?) stateless, so describing in terms of data condenses test code to essential inputs and outputs
Using a data representation makes tests language agnostic and reusable in target languages
Representing tests as data would mirror the essence of user inputs and outputs described in the wrapping layer
A concise notation would incentivize uniformity and concision of testing
Cons:
Need to create a test harness/mini-framework to parse and execute test cases written as data
Target languages wanting to reuse tests-as-data on ICU4X client libraries would need to write a native harness/mini-framework
Tests that depend on stateful resources (ex: data stores) with setup/teardown phases would have to be handled in some other way
Representing tests in a new way would require some initial adjustment
Existing libraries
Most searches for "data driven testing" produce results for databases, spreadsheets, and automated web UI testing. Links to more relevant pre-existing libraries are welcome.
Some examples of test libraries written to reduce the cognitive load when testing, especially when testing data collections:
I'm merging #223 and #410 into this issue. Some notes from those issues:
We want the data to come through the data provider, but we also want data that lives close to the test. Can we maybe put invariant (und) data in the test file and load resources for other locales?
Is there a way to make the test data extensible by users?
One part of the larger testing strategy for ICU4X would be to have a clean, consistent way of organizing the unit tests for the business logic (i18n algorithms). In particular, it would be nice to have a data-oriented style of testing, as exemplified already in some parts of @zbraniecki's unic-locale repo. ICU unit tests tend to be written in a parameterized style, but the idea here is to take the data-driven nature further.
Pros and Cons
Pros:
Cons:
Existing libraries
Most searches for "data driven testing" produce results for databases, spreadsheets, and automated web UI testing. Links to more relevant pre-existing libraries are welcome.
Some examples of test libraries written to reduce the cognitive load when testing, especially when testing data collections:
Testing features
Beyond just asserting that the actual return value matches the provided expected value, we should also consider the following testing aspects: