The full scope of the metriq-client integration (i.e. the ability to dispatch and update Metriq.info) is likely beyond this one issue and will almost surely change. Still, the general structure and scaffolding necessary to be in place to support this workflow can be achieved.
The content below is extracted from the discussion in #32:
We don't have much in the way of Metriq and Metriq-client integration, but we will likely want to subdivide some of the "client-specific" components into its directory. Something like:
metriq_upload_result.py: Taking a BenchJobResult and associated Metriq submission information and dissecting the relevant components to be uploaded to Metriq.info.
metriq_metadata.py contains the mapping of submission, result, etc. IDs on Metriq.info to specific benchmarks that we run. This is not exhaustive and needs to be updated as new results, submissions, etc., get added to the platform.
The full scope of the metriq-client integration (i.e. the ability to dispatch and update Metriq.info) is likely beyond this one issue and will almost surely change. Still, the general structure and scaffolding necessary to be in place to support this workflow can be achieved.
The content below is extracted from the discussion in #32:
We don't have much in the way of Metriq and Metriq-client integration, but we will likely want to subdivide some of the "client-specific" components into its directory. Something like:
Where:
metriq_upload_result.py: Taking aBenchJobResultand associated Metriq submission information and dissecting the relevant components to be uploaded to Metriq.info.metriq_metadata.pycontains the mapping of submission, result, etc. IDs on Metriq.info to specific benchmarks that we run. This is not exhaustive and needs to be updated as new results, submissions, etc., get added to the platform.