-
See: https://github.com/facebookresearch/metaseq/blob/9afea52f5988fcbfe6133591fc27dd56044bd4ea/metaseq/checkpoint_utils.py#L424-L425
Previous attempt at removing this hack broke generation, evals, …
-
```
complete_history = []
try:
complete_history = joblib.load(filename)
#evals_loaded_trials = len(trials.statuses())
#max_evals += evals_loaded_trials
…
-
command:
accelerate launch run_evals_accelerate.py --model_args="Llama-2-7b-chat-hf-8bit,quantization_config="load_in_8bit=True"" --tasks "helm|hellaswag|1|0" -- --output_dir ./evalscratch
Resul…
-
### Describe the bug
Hi,
I got the following error when my PR is checked. Here is the [link](https://github.com/openai/evals/pull/651) for my PR request. Does anyone know what is happening here? I…
-
Local evals take quite a long time, about 2 minutes on my rather fast machine. Often times however, you will have already eval'd some revision before. For example, running two reviews against master a…
Atemu updated
10 months ago
-
https://www.nytimes.com/2024/02/01/technology/perplexity-search-ai-google.html
> Perplexity also allows users to search within a specific set of sources, such as academic papers, YouTube videos or …
-
I think we can avoid our heavy use of eval and Bass.invokelatest if we use `@generated` functions.
-
### Describe the feature or improvement you're requesting
[build_eval.md](https://github.com/openai/evals/blob/main/docs/build-eval.md#for-model-graded-evals-a-step-by-step-workflow) says:
> In g…
-
Each panel in the dashboard has a hard-coded eval to extract as human-readable time from epoch time. Build a macro that can accomplish this and update the dashboards to use it.
-
Hi! Thanks for sharing the great work!
I have noticed that we can get `meta_data`, which contains the hand pose in terms of Bones, from the `.h5` files.
And there are MANO meshes in the `evals` fold…