distributed-system-analysis / pbench

A benchmarking and performance analysis framework
http://distributed-system-analysis.github.io/pbench/
GNU General Public License v3.0
188 stars 108 forks source link

Allow typed metadata on upload #3566

Closed dbutenhof closed 1 year ago

dbutenhof commented 1 year ago

PBENCH-1283

Normally metadata keys are set through PUT /datasets/<id>/metadata using an application/json request body, allowing the full expressiveness of JSON, specifically to specify sub-objects, integers, floating point numbers, and boolean literals.

I added the ?metadata query parameter on PUT /upload/<name> to allow a client to specify metadata during the upload instead of waiting; however the nature of the HTTP interface constrains these to text strings, which can be limiting.

This PR generalizes the typed "metadata expression" syntax developed to support generalized metadata ?filter expressions on GET /datasets to allow both quoting value strings and typing them. For example, one could now set a JSON value using

?metadata=global.legacy.pbench:'{"host":"agent","migrate":true}':json

We support str, int, float, bool, and json types; e.g.,

?metadata=global.tester.run_id:100:int,global.tester.good:true:bool

(P.S.: when updating the upload.md I noticed that I'd somehow missed ever writing a metadata.md for the /datasets/<id>/metadata APIs, and I corrected that oversight. I think we're now also missing visualize and compare API documentation, but that's a bit too far disconnected from this topic.)