unitaryfund / metriq-client

Fork of papers-with-code client
https://metriq.info
Apache License 2.0
3 stars 9 forks source link

Benchmarks: Port core functionality of benchmark runner into client project #96

Open vprusso opened 1 year ago

vprusso commented 1 year ago

It seems to me that it makes the most logical sense to put the functionality of running the benchmarks into the client project (let me know if you disagree, @WrathfulSpatula).

At present, the base-level functionality for running the benchmarking pipeline is located here: https://github.com/unitaryfund/metriq-api/blob/main/benchmark/run.py

We should create a folder in the root directory with the name benchmark/ with the contents of run.py contained within as a first step. We should also ensure that the benchmarking pipeline is able to be run as well.

@srilakshmip03 / @QuantumAli I think focusing work on the benchmarks would take priority for the client, so it probably makes sense to sync on this for further details.