Ability to stop the test if score goes below a certain value.

[pawanrawal]

Add ability to end test if score goes below a certain value. This value can be part of config which can be shared during the authenticate call.

[mohitranka]

I don't understand why this is needed?

1. Since the order of hard/easy questions would be random -- it is unfair to the candidate who get the harder bunch of the question in the initially.
2. What is the point of it even? Traditional f2f interviews have early termination of the interview process because human/engineers' time is expensive -- so if a process gets stopped in 2 hours instead of say 5 hours times n, it saves lots of $$$ for the company, but that would not be the case with scalable approaches like Gru, because no candidate would mind spending the full hour instead of early termination to save his 15-20 minutes and it does not really make a difference to the interviewers from the cost point of view.

[manishrjain]

The tests should start with easy questions and only go towards harder ones if the candidates correctly answer them.

What we're saving is spilling out all questions to a candidate who isn't serious about the interview. For e.g., say all they want to do is to publish the questions, so they just answer option A for all the questions. If we can detect that a lot of their answers are just wrong, there's no point in giving all the questions. We should just protect our questions and stop the test.

Note that a lot of engineering hours go into creating these questions. We don't pick them up from the internet -- we have to create them from concepts.

[mohitranka]

Makes sense. Thanks.

[mohitranka]

Picking up this task.

[mohitranka]

Design thoughts:

1. By default, the early termination is not applicable.
2. Since the score at the start of the quiz is 0, we can either keep the threshold as either the low average score (say 0.x of total max available points till the current point in the quiz) or keep it at a constant negative threshold. Since the purpose of this feature is to avoid malicious brute-forcing, we can pick a constant negative number.
3. The decision of quiz termination is made at the point of score calculation after every question.

Thoughts? @dgraph-io/team.

[pawanrawal]

I think keeping it at a constant negative number makes sense. Other points also look good.

[pawanrawal]

Closing this for now. We can bring this up again when we make the quiz adaptive.