-
when I prepare the environment by using pip24.2 all package successfully installed except torchsde:
pip install torchsde==0.2.5
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/
Collec…
-
Whenever I use a LM to evaluate something I ask a Yes/No question and do `logprob_yes - logprob_no`. This worked very well for me in the past. How can I access logprobs in my metric function?
-
I evaluated llama3-8b-Instruct using the gsm8k benchmark, and found some interesting phenomenons.
1. Huggingface and vllm has similar results
![image](https://github.com/EleutherAI/lm-evaluation-h…
-
```
When I use interpolate-ngram to interpolate two models by CM or GLI with
perplexity optimization, I get following faults:
1st:
interpolate-ngram -lm "model1.lm, model2.lm" -smoothing ModKN -inte…
-
Drawer (show/hide applets) it's not working.
It used to work great in older versions and did the job very well.
* Applet version/Build date: "last-edited": 1683827772
* Cinnamon ver…
-
FYI looks like ESPN changed their API again. You may need to update to hit `https://lm-api-reads.fantasy.espn.com/apis/v3/games/ffl/seasons/`
(I'm working on a JVM implementation of a similar libra…
-
The following setup in the TokenEnforcer initialization cause any custom CharacterLevelParserConfigs to be overriden with default values
https://github.com/noamgat/lm-format-enforcer/blob/fe6cbf107…
-
### Which API Provider are you using?
Ollama
### Which Model are you using?
Llama 3.2: 8b
### What happened?
I am using the default url with Ollama and Llama 3.2: 8B to generate some basic CRUD f…
-
`[ Linux | CUDA 12.2 | py 3.11 ]`
### Describe the bug
Hi, thanks for your awesome work!
When infering `Llama3.1/70B-Instruct/6.0bpw` over the OAI API, it throws:
```sh
INFO: 127.0.0.…
-
Hi, It seems that the same code is **working fine with when the Megatron-LM that I git-cloned in April. With the latest Megatron-LM, I've got the following error raised with the pretrain_gpt.py code. …