quintel / mechanical_turk

Automatic tester for etengine
1 stars 0 forks source link

Multi scenario specs #160

Closed mabijkerk closed 2 years ago

mabijkerk commented 2 years ago

This branch adds new functionalities to the Mechanical Turk, such as the softly_equal functions to test queries, and the possibility to test a suite of existing scenarios.

In addition some specs for the 'Laws of nature' milestone have been added in to branch.

The primary demand spec does not yet work completely, due to underlying issues described in #148. The failing checks have been set to pending. This way, the branch can be merged to master and the new functionalities can become generally available. Any fixes to existing specs or new specs should then be made on new separate branches.

@noracato could you verify whether all the specs (with the exception of primary demand) pass? This branch relies on queries that are added to the mechanical-turk-queries branch on ETSource, see https://github.com/quintel/etsource/pull/2631.