search

JdeRobot / PyOnBrowser

0 stars 2 forks source link

framework for automated regression tests #4

Open paurea opened 4 years ago

paurea commented 4 years ago

It would be interesting to create a framework for automated tests in the command line. There are three components needed for this:

1) EHAL: an emulated HAL, with fake sensors which get their output from a file with measurements. 2) A standard way to add new tests with their right output (regular output and errors if there should be any). For example, a directory tests where every test_zzz*.py file is a test and zzz.okout file are the right output (maybe errors separated from console outputs) and sensorY_zzz.input files 3) A shell script (or Python) which for each tests runs it and compares the output with okout and prints an error if they are different. 4) A good set of tests which works out the supported language one feature at a time. This is already started, but more need to be added. 5) When this is working, issues w.r.t. the translator can be accompanied with a test .py, inputs and maybe an expected output.

I'll get on this as soon as I can. If anyone else starts with this, keep me in the loop.

paurea commented 4 years ago

I added 1) (the two commits EHAL and EHALwork), sorry I pushed by mistake. You and add the parameter --emulate /bla/dir with files for input in sensors. Will document it and continue to 2).

paurea commented 4 years ago

@rinz13r take a look at it to make sure I didn't blunder anything as you are more familiar with it.

paurea commented 4 years ago

I am done with 3. 4 still to be done. Also, more support for EHAL from someone familiar with the robots.

paurea commented 4 years ago

Added A/B testing, which is complementary to the integration/regression tests. See the last part of the README for details.