-
Thanks for your amazing work! I have two questions regarding your implementation on the TAL task.
1. The TAL tasks require the model not only to predict the start/end times of candidate predictions…
-
1. The current implementation `reinterpret_cast`s between `node_type` and `json_type`. `reinterpret_cast` is very likely to (although not always) result in UB here, and it always breaks constant evalu…
-
When running multiple instances of Timeloop in parallel, I get this error:
```
execute:accelergy evaluation/features/features.8/Conv/eyeriss_like.yaml --oprefix timeloop-mapper. -o ./ > timeloop-m…
-
## TODO
**1st iteration**
- [x] Dump the assessments into the `evaluation.csv` every time a task is executed
**2nd iteration**
- [x] Create the other CSVs from the `evaluation.csv`
- read…
-
According to the State of the art model [evaluation](https://paperswithcode.com/sota/object-detection-on-coco) in papers with code, Transformer based object detectors provide better `box mAP` that yol…
-
I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
I am trying to run the template code from the Github ReadMe page.…
-
If I interpret the https://arxiv.org/pdf/2404.16790 paper correctly, you have used transit maps for evaluation. However, I can't figure out which maps have been used: neither from the GitHub repo, nor…
-
INFO:mteb.evaluation.MTEB:
## Evaluating 1 tasks:
─────────────────────────────── Selected tasks ────────────────────────────────
Retrieval
- MSMARCOv2, s2p
INFO:mteb.evaluation.MTEB:
…
-
### Overview
The Statsig provider [converts](https://github.com/open-feature/go-sdk-contrib/blob/main/providers/statsig/pkg/provider.go#L391) [evaluation context](https://openfeature.dev/specificat…
-
Hi team! First things first, thank you for creating this wonderful benchmark!
I believe its curation and evaluation required a lot of effort, so I really appreciate it that you open-sourced the data…