Tracking: Test generator MVP

[mk-mxp]

Test generator MVP to use after configlet generate for practice exercises:

• [x] #638 Integrate POC test generator into new directory contrib/generator, so it is not installed in CI
• [x] #638 Add a parameter to choose the exercise by slug
• [x] #638 Use local configlet cached problem specification repository as information source
• [ ] #663 Do not do anything, if exercise has no canonical-data (there are some)
• [ ] #663 Use /exercises/practice/${slug}/.meta/config.json>files>test and check that there is only one entry
• [x] #638 Write tests to already created /exercises/practice/${slug}
• [x] #638 Have test class skeleton
• [ ] #663 Have test class DocBlock with "PHP track rules"
• [x] #638 Have testdox with description
• [x] #638 Have UUID in DocBlock
• [x] #638 Have all tests marked incomplete
• [ ] #663 Have $input, $expected
• [ ] #663 Have $subject = new ${PascalSlug};, $actual = $subject->${property}($input); and assertSame($expected, $actual);
• [ ] #663 Handle unknown data, so the user can act on it
• [ ] #663 Handle nested groups of tests
• [ ] Bonus: Add composer script to handle exercise generation from scratch. The problem is handling the slug required for exercise generation. That requires some additional scripting, so that only those commands get the slug that need it and the others do not fail.
• [ ] #663 adds unit tests, but no integration tests Bonus: Add golden tests for test generation of some exercises
• [ ] #663 handles unknow data instead Bonus: Add JSON validation using the configlet scheme (if that exists), else at least field existance and character sets
• [ ] Bonus: Add PHP CodeSniffer (with rules for test-generator only) and more static analysis

Not MVP:

• Producing production ready tests
• scenarios
• Reimplementation
• Ignoring tests marked with include = false in tests.toml

Open questions:

• Should it be a single command app or do we want to add more commands like update-exercise (or so)?
• Should we use another code producer library like laminas/laminas-code?

[mk-mxp]

@tomasnorre Did you add some functionality to your local copy of the test generator that you would want to have here, too? I think I'll find time on the weekend to take some of the steps outlined here and would be happy to integrate the "latest" version.

[tomasnorre]

@mk-mxp I don't have anything locally, that is not already published here https://github.com/tomasnorre/exercism-tests-generation