Allow setting raise_on_mismatches to base class level for tests

[mtoygar]

We are using a base experiment class to manage our publish logic. All the actual experiments are extending from this base class. However, if we set raise_on_mismatches to this base class the result doesn't take effect as the method is using class instance variables(instead of class variables). In the end, people occasionally forget to use this test helper for their newly created experiments.

It would be great if we can manage this test helper via a base class. To elaborate more on this, this is somewhat our structure;

class BaseExperiment
  include Scientist::Experiment

  def initialize
    # setting up some instance variables mainly needed for publish logic
  end

  def publish(result)
    # some custom logic
  end
end

class WidgetExperiment < BaseExperiment
  def initialize
    # setting up custom variables
    super
  end

  def enabled?
    # custom enabled logic
  end
end

class AnotherExperiment < BaseExperiment
  # similar context with widget experiment
end

What we want is to use BaseExperiment.raise_on_mismatches = true to ensure all child experiments are tested on tests without needed a separate test setup. Would such a need make sense to you?

[isouravgope]

We are using a base experiment class to manage our publish logic. All the actual experiments are extending from this base class. However, if we set raise_on_mismatches to this base class the result doesn't take effect as the method is using class instance variables(instead of class variables). In the end, people occasionally forget to use this test helper for their newly created experiments.

It would be great if we can manage this test helper via a base class. To elaborate more on this, this is somewhat our structure;

class BaseExperiment
  include Scientist::Experiment

  def initialize
    # setting up some instance variables mainly needed for publish logic
  end

  def publish(result)
    # some custom logic
  end
end

class WidgetExperiment < BaseExperiment
  def initialize
    # setting up custom variables
    super
  end

  def enabled?
    # custom enabled logic
  end
end

class AnotherExperiment < BaseExperiment
  # similar context with widget experiment
end

What we want is to use BaseExperiment.raise_on_mismatches = true to ensure all child experiments are tested on tests without needed a separate test setup. Would such a need make sense to you?

definitely makes sense