Open zhu0619 opened 1 year ago
Interesting! I haven't thought a lot about generative benchmarks yet, but it seems to me this would be so different we would end up introducing a new BenchmarkSpecification
subclass (see e.g. also #5) that would maybe be better suited to store this information.
Context
45 reminded the fact the "optimization direction" or "optimal value range" is important to benchmark for generative model, such as molecular generation and optimization.
Description
Currently, this optimization direction and thresholds are handled in
user_attributes
.Acceptance Criteria
direction
,threshold
oropt_interval
are added to the ColumnAnnotation class.This is a low priority task, maybe for V1?