-
Hi there,
I appreciate your work on AP-BWE. The results and samples provided are very impressive. I'm interested in evaluating the performance of your model for my research.
Could you release th…
-
**Describe the bug**
I encounter 'ValueError: Evaluation LLM outputted an invalid JSON. Please use a better evaluation model.' while using most popular open source chat models in DeepEval framework. …
-
- Outage probability
- Number of steps: print accuracy chart
- [score function](https://keras.io/getting-started/sequential-model-guide/)
-
Hi All ,
Wanted to understand , when we evaluate using the plot_decision_boundary. But how do we test the model apart from it.
Do we give the new test values to the model and then plot it again…
-
### System Info
I am running on A100 with 40 GB GPU memory
### Who can help?
@SunMarc and @younesbelkada
### Information
- [ ] The official example scripts
- [X] My own modified scri…
-
DoLa decoding on Mixtral model with multi-GPU setup returns error:
```
Traceback (most recent call last):
File "~/src/evaluation/test.py", line 8, in
generate_ids = model.generate(inputs.in…
-
### Open-weight models:
- HugingFace Model ID: princeton-nlp/gemma-2-9b-it-DPO, princeton-nlp/gemma-2-9b-it-SimPO
- Pretty Name: gemma-2-9b-it-DPO, gemma-2-9b-it-SimPO
- Supported by vLLM: yes
#…
-
-
학습이 잘 되지 않음. image가 크지않아서 일 수도 있다는 생각이 들긴함.
처음 learning rate를 adam(lr=0.00001)을 두고 돌렸는데 너무 학습이 느린가라고 생각돼서 30 epochs만 확인하고 10배 키운 후 다기 500 epochs 확인. 그러나 거의 학습되지않음
-
Is it possible to provide more details about the MATH benchmark evaluation https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/eval_details.md#math?
E.g. would be great to know ho…