-
**Describe the feature**
Provide unified, fast evaluation script for Benchmark using unified datasets #11 and standard dataloaders #14
Evaluation metrics:
- [x] NLU: Acc, dialog act tuple P/R/F1
…
-
Thanks for sharing your code.
I try to reproduce your results, I found in the demo there are some .txt files "st_peters_.txt". l did not find a way to generate them.
I will be appreciated if you inf…
-
Thank you for your excellent work. I'm studying the evaluation methods and I'm insterested in your code, and the figures of correlation of meatures. Could you please provide me the code for more metri…
-
I'm trying to understand how evaluation are done for L1 difficulty. Essentially L1 difficulty selects a subset of GT boxes. In this case, I'm not sure how are precision evaluated. Specifically, I wond…
-
In your article, you wrote:
For the evaluation on real-world datasets without ground truth, we employ the widely-used non-reference perceptual metrics: FID and NIQE.
Is FID a non-reference indicator…
-
Previous backgammon engines have calculated various metrics to help them with their evaluation: pip counts, number of blots, etc.
Many of those metrics were introduced by Hans Berliner: https://www…
-
### Description:
The prometheus.yaml configuration given in the below page is incorrect in terms of its indentation.
https://mg.docs.wso2.com/en/latest/how-tos/observability/
### Steps to repro…
-
Hello, I would like to run your code. A simple demo or sample script or inference on a complete data set(, etc.) is fine. Could you tell me the corresponding running command or method?
Thanks in ad…
lzt02 updated
3 months ago
-
Hi!
I noticed an inconsistency in the calculation of the Metric in train.py.
The `lpips()` function is used without specifying the net_type option, as seen here:
https://github.com/ShenhanQian/Gaus…
-
### System Info
transformers=4.40.1
pytorch=2.2.1
deepspeed=0.14.1
accelerate=0.29.0
### Who can help?
@pacman100 Hello, I tried fine-tuning LLMs using transformers' trainer with deepspeed…
getao updated
3 weeks ago