Similar function signatures for Learning Curve retrieval

[Neeratyoy]

In my opinion, having the same function signatures for the two types of retrieval makes sense:

• query(config: C, fidelity) -> R: gives the point or the exact result at fidelity
• trajectory(config: C, fidelity) -> List[R]: gives the full learning curve until fidelity

Additionally, I was wondering if we can already have a placeholder parameter that can be further passed to either a benchmark state during initialisation or to these function calls above. That is whether the evaluation is being continued or thawed. The only thing that should change then is the cost of continuations. Not sure where then is the best place to change. We could also kind of do it post-hoc for the evaluations made by subtracting the cost incurred for a lower fidelity evaluation.

For the non-benchmark case, NePS anyways handles continuation so should work out fine I believe.

@DaStoll @eddiebergman thoughts? Please feel free to take the call.

[eddiebergman]

As a note, I changed def trajectory to be but I see your argument none the less

# C, the config type for the benchmark
# F, the fidelity type for the benchmark (int, float)
# R, the result type for the benchmark
@abstractmethod
def trajectory(
    self,
    config: C,
    *,
    frm: F | None = None,  # None indicates min of fidelity
    to: F | None = None,   # None indicates max of fidelity
) -> list[R]:

[eddiebergman]

Regarding the freeze-thaw action, it only makes sense with "freeze-thawable" algorithms, I would leave it to post-hoc for simplicity of the benchmark