github / scientist

:microscope: A Ruby library for carefully refactoring critical paths.
MIT License
7.47k stars 442 forks source link

Add support for experiment after_run blocks #211

Closed nickh closed 1 year ago

nickh commented 1 year ago

This change allows an experiment to run custom behavior based on experiment results, eg

science("my-experiment") do |e|
  e.use { control_method }
  e.try { candidate_method |

  # Slower candidates need further investigation
  e.after_run do |result|
    control_duration = result.control.duration
    candidate_duration = result.candidates.first.duration

    if candidate_duration - control_duration > 15
      Logger.debug("Slow experiment candidate #{context}")
    end
  end
end