kazk
commented
3 years ago I'd like to eventually automate some mechanical fixes/changes. We have tests, so we can apply the change and make sure it's still valid in many cases. There are many existing tools, and we can maintain some of our own for each language. For example, there are many tools for JavaScript that transforms ASTs (transpiling is common). I made a PoC few years ago that can translate tests to Mocha, but I postponed the idea after seeing many kata using the test framework in very unexpected ways, and it was non-trivial to covert them.
But I'm strongly against automating translations. Qualified does support generating tests from data for many languages and test frameworks already, but it's not a great experience on both sides (authors and candidates). If interested, there's a blog from 4 years ago. It sounds great in theory, but it's not practical and leads to bad UX because languages are different. You'll get many languages supported, but none of them will be great (e.g., awkward types, arbitrary requirements that makes no sense, poor test feedback, etc.). You can only test serializable inputs and outputs too.
If something changes about a language, change the code generation, run it again. No more lists of 3800 kata to update.
What about 3800 reference solutions? If we're only generating tests, then we rarely need to update anyway.
MANY kata would become available in MOST languages. Or rather, after having been translated to the one language to rule them all.
If we valued this, we'll be doing challenges based on stdin/stdout like many others. Codewars was born because Jake didn't like that. I'd rather have one idiomatic translation than 50 half assed generated translations. Translation is great for learning too (we do need a better review system and let more experienced users to give feedback (mentoring)).
namedots
Big brain idea here. You're guaranteed to have considered it, nothing new, but I wish to talk about it.
There's this list. https://github.com/codewars/content-issues/wiki/List-of-Python-Kata-to-Update It's pretty long. The task to carry out is mechanical unless one makes additional improvements.
Most of these kata have json-like input and output. This makes them pretty much all the same. No human should be touching them. They can be automatically generated with the help of information about kata.
If something changes about a language, change the code generation, run it again. No more lists of 3800 kata to update.
The tests can be consistent across languages. Bugs get fixed in one place, not 5 or twenty. For performance requirements, the program that generates tests and checks answers could receive parameters to tailor size of tests.
MANY kata would become available in MOST languages. Or rather, after having been translated to the one language to rule them all.
So when one authors a kata, one would write this test generator, plus some meta data, some sort of schema. That would then be used to generate inital code and tests at the press of a button.
Obviously, this would be non-trivial. It would change kata authoring editor UI, and the backend would have to support running two languages. (static compilation might help, at the cost of limiting which languages can serve as testers) (sending it over network could be an option too, the data is typically small) Writing translation generators is a project all on its own. Figuring out the metadata format is somewhat of a challenge.
No clue whether you do translations over at Qualified. I suppose that would be the real selling point.