Move all existing benchmark configs to the benchmark_metadata collection in our DB
Existing API endpoints GET /benchmarkconfigs and GET /benchmark/{benchmark_id} have the same behaviors as before, except 1. their data now comes from DB 2. The backend now checks whether the user has read access.
Schemas
Added BenchmarkCreateProps and BenchmarkUpdateProps as POST and PATCH bodies. Copied all properties from the original BenchmarkConfigs to BenchmarkCreateProps and marked some as nullable.
Added shared_user as a required property in the create props and optional in the update props.
Added creator, created_at, and last_modified properties to BenchmarkConfigs in DB. Removed the hack thanks to #510
Renamed visibility to is_private in favor of simplicity and consistency with system metadata.
New API endpoints
POST /benchmark: Uses custom logic to reject a benchmark if its parent is empty and has views as None.
PATCH /benchmark/{benchmark_id}: Requires write access. All fields of a benchmark are updatable except id, creator, created_at, and last_modified. During update, any fields with None values in the update props is skipped to prevent overwriting the original DB document.
Modify the CLI and web UI to support benchmark CRUD operations.
Add benchmark collection in prod DB after merged
Future TODO
Add pagination or caching to benchmark_configs_get() as it currently retrieves all parent benchmarks in DB when the parent argument is "". This is not an issue for now as we only have a few benchmarks.
Support the storing of benchmark logos and potentially migrate from S3 buckets to Google Cloud.
Resolves #425 (Part 1 of 2)
benchmark_metadatacollection in our DB/benchmarkconfigsand GET/benchmark/{benchmark_id}have the same behaviors as before, except 1. their data now comes from DB 2. The backend now checks whether the user has read access.BenchmarkCreatePropsandBenchmarkUpdatePropsas POST and PATCH bodies. Copied all properties from the originalBenchmarkConfigstoBenchmarkCreatePropsand marked some as nullable.shared_useras a required property in the create props and optional in the update props.creator,created_at, andlast_modifiedproperties toBenchmarkConfigsin DB. Removed the hack thanks to #510visibilitytois_privatein favor of simplicity and consistency with system metadata./benchmark: Uses custom logic to reject a benchmark if itsparentis empty and hasviewsas None./benchmark/{benchmark_id}: Requires write access. All fields of a benchmark are updatable exceptid,creator,created_at, andlast_modified. During update, any fields with None values in the update props is skipped to prevent overwriting the original DB document./benchmark/{benchmark_id}: Requires write access.Next steps
Future TODO
benchmark_configs_get()as it currently retrieves all parent benchmarks in DB when theparentargument is"". This is not an issue for now as we only have a few benchmarks.