davezych / shience

A .NET port(ish) of Github's Scientist library. (https://github.com/github/scientist)
MIT License
9 stars 1 forks source link

Implement Before Run functionality #18

Closed davezych closed 8 years ago

davezych commented 8 years ago

If a candidate test requires setup that shouldn't be included in runtime calculations, there needs to be a way to specify setup code that will run only if the candidate is going to be run.

var controlParameter = GetParam();
var candidateParameter = GetExpensiveCandidateParam();

science.Test(() => SomeSimpleCall(), () => SomeMoreComplexCall(candidateParameter))
            .Execute();

In this instance, the expensive call will always be run, which doesn't make sense if the experiment is enabled for only a fraction of requests. Something like this would be better:

var controlParameter = GetParam();
object candidateParameter = null; //Obviously object is a terrible type

science.Test(() => SomeSimpleCall(), () => SomeMoreComplexCall(candidateParameter))
            .BeforeCandidateRun(() => candidateParameter = GetExpensiveCandidateParam()) //Assuming closure rules allow this
            .Execute();

I don't believe this is needed for the control test, because that should always be run, regardless of experiment enabled percentage.

MovGP0 commented 8 years ago

this one doesn't need a change at all. you only need to write:

Shience.New<TResult>()
    .Test(SomeSimpleCall, () => SomeMoreComplexCall(GetExpensiveCandidateParameter()))
    .Execute();
davezych commented 8 years ago

Yep, you're right.