The public_benchmark_identifiers method currently returns all identifiers in the BenchmarkType table, which include parent Benchmarks that are not meant to be implemented in the codebase or be added to the benchmark registry. For example, the method would return average_vision and neural_vision as benchmarks, which are not concrete benchmark instances, leading to failures when trying to score submitted models on them. Therefore, this PR changes the method to return all public identifiers from the BenchmarkInstance table and adds a unit test to ensure the intended behavior is captured.
The
public_benchmark_identifiersmethod currently returns all identifiers in theBenchmarkTypetable, which include parent Benchmarks that are not meant to be implemented in the codebase or be added to the benchmark registry. For example, the method would returnaverage_visionandneural_visionas benchmarks, which are not concrete benchmark instances, leading to failures when trying to score submitted models on them. Therefore, this PR changes the method to return all public identifiers from theBenchmarkInstancetable and adds a unit test to ensure the intended behavior is captured.