search

partiql / partiql-tests

Test data for checking conformance to the PartiQL specification.
Apache License 2.0
7 stars 1 forks source link

Port `EvaluatorTestSuite.kt` tests #28

Closed alancai98 closed 2 years ago

alancai98 commented 2 years ago

8 -- ports tests from partiql-lang-kotlin's EvaluatorTestSuite.kt.

Tests ported using the following process:

  1. Use script that converts EvaluatorTestSuite.kt tests to the evaluation conformance test format
    • Script reads data from the Kotlin test data class and stores as Ion
    • Reformats some test fields (e.g. $partiql_bag -> $bag, remove new lines)
  2. Move tests and environments to separate test files
  3. Document tests in partiql-tests that may need omitted or need to be looked at before adding. E.g. due to not in spec or different compile/evaluation option

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

alancai98 commented 2 years ago

Following #25 (defines the evaluation modes), will need to update the ported tests and include the additional evalMode in the assertions.

Following #30 (ports the partiql spec tests + restructures eval test directories), will also need to update the test directories.

am357 commented 2 years ago

8 -- ports tests from partiql-lang-kotlin's EvaluatorTestSuite.kt.

Tests ported using the following process:

1. Use [script](https://github.com/alancai98/partiql-lang-kotlin/blob/eval-test-converter/lang/test/org/partiql/lang/eval/EvaluatorTestSuiteToIon.kt) that converts EvaluatorTestSuite.kt tests to the evaluation conformance test format

* Script reads data from the Kotlin test data class and stores as Ion

* Reformats some test fields (e.g. `$partiql_bag` -> `$bag`, remove new lines)

2. Move tests and environments to separate test files

3. Document tests in `partiql-tests` that may need omitted or need to be looked at before adding. E.g. due to not in spec or different compile/evaluation option

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Curious have you then manually split evaluator_test_suite.ion output from the script?

alancai98 commented 2 years ago

Curious have you then manually split evaluator_test_suite.ion output from the script?

Yeah, the script originally ported the EvaluatorTestSuite.kt tests to one file. I decided to move the tests to separate files by the namespace/group for better organization. It wasn't too much manual effort other than checking the evaluation environments were in the correct locations.

alancai98 commented 2 years ago

Yeah, totally agree that it's hard to do any verification of these tests in these reviews alone. I'm working on https://github.com/partiql/partiql-tests/issues/33 to create a test runner in Kotlin to validate the tests are ported correctly.