search

apertium / apertium-regtest

Regression testing system for Apertium language data and translators
https://wiki.apertium.org/wiki/Apertium-regtest
GNU General Public License v3.0
1 stars 0 forks source link

Can we have generated and editable files in different folders? #2

Closed unhammer closed 3 years ago

unhammer commented 3 years ago

Currently there are 165 files in nno-nob/test for 6 testsets. If I want to add inputs/golds, that's quite a lot of file names for my human eyes to parse. Would it be possible to have something like

/test/humaneditable.input.txt
/test/humaneditable.gold.txt
/test/generated/pendingtaggeroutputexpected.txt

?

(In my quickly-hacked-together regtest approximation in nno-nob/tests I do this, with 20 different test sets)

TinoDidriksen commented 3 years ago

That makes sense.

I'd say go even further, keeping input files in test/ directly, and then having test/expected/ test/gold/ test/output/ all separated. Because there can be a gold file for each step in the pipe, they would likewise flood the main folder.

And having output/ separate and not versioned makes it very easy to .gitignore and clean

mr-martian commented 3 years ago

With the current setup, tests.json can specify an arbitrary (relative) path for the input, so those at least could be put in a different folder.

Having subfolders rather than different filenames does seem like a much better system and I wish I had thought of it before setting it up in 365 places.

I suppose there could be a setting in tests.json to specify the directory structure.

mr-martian commented 3 years ago

Done, and the tools/flat2nest.py script will automate the transition.