Consider the QRC algorithm implementation where we assume beta=1.0 by default. We execute an experiment like:
from ml_experiment.DefinitionPart import DefinitionPart
part = DefinitionPart(name='qrc')
part.add_sweepable_property('alpha', (2**-i for i in range(5, 9)))
part.commit()
then later we decide to sweep beta:
from ml_experiment.DefinitionPart import DefinitionPart
part = DefinitionPart(name='qrc')
part.add_sweepable_property('alpha', (2**-i for i in range(5, 9)))
part.add_sweepable_property('beta', [0.5, 1.0, 2.0])
part.commit()
A problem occurs where every result is now considered invalidated, despite the fact that we already have results for the beta=1.0 case from the first experiment execution.
This PR addresses exactly this issue. The PR is structured in a sequence of 3 commits, it might be useful to go commit-by-commit (i.e. in order to see where the Maybe monad fits in).
You can test this PR with the following (run once to generate the v0 table, uncomment and generate the v1):
from ml_experiment.DefinitionPart import DefinitionPart
part = DefinitionPart(name='qrc')
part.add_sweepable_property('alpha', (2**-i for i in range(5, 9)))
# part.add_sweepable_property(
# 'beta', [0.5, 1.0, 2.0],
# assume_prior_value=1.0,
# )
part.commit()
We expect the beta=1.0 rows to have ids 0, 1, 2, 3 and the full range of the v1 table to be [0, 11]. The original implementation would have ids [4, 15].
Consider the QRC algorithm implementation where we assume
beta=1.0
by default. We execute an experiment like:then later we decide to sweep
beta
:A problem occurs where every result is now considered invalidated, despite the fact that we already have results for the
beta=1.0
case from the first experiment execution.This PR addresses exactly this issue. The PR is structured in a sequence of 3 commits, it might be useful to go commit-by-commit (i.e. in order to see where the Maybe monad fits in).
You can test this PR with the following (run once to generate the v0 table, uncomment and generate the v1):
We expect the
beta=1.0
rows to have ids0, 1, 2, 3
and the full range of the v1 table to be[0, 11]
. The original implementation would have ids[4, 15]
.