lm_eval: error: unrecognized arguments: --wandb_args

[TheRealM4rtin]

Hi, I'm getting this error when trying to run lm_eval with wandb arguments. I followed the correct installation, and can see the argument in lm-evaluation-harness/lm_eval/ main .py.

usage: lm_eval [-h] [--model MODEL] [--tasks task1,task2] [--model_args MODEL_ARGS] [--num_fewshot N] [--batch_size auto|auto:N|N] [--max_batch_size N] [--device DEVICE] [--output_path DIR|DIR[/file.json](http://localhost:8888/file.json)] [--limit N|0<N<1] [--use_cache DIR] [--decontamination_ngrams_path DECONTAMINATION_NGRAMS_PATH] [--check_integrity] [--decontamination_ngrams_path DECONTAMINATION_NGRAMS_PATH] [--check_integrity] [--write_out] [--log_samples] [--show_config] [--include_path DIR] [--gen_kwargs GEN_KWARGS] [--verbosity CRITICAL|NAMS_PATH]. [--verbosity CRITICAL|ERROR|WARNING|INFO|DEBUG] lm_eval : error : unrecognized arguments : --wandb_args project=llm-compression-project

But when I type lm_eval --help, I don't see any wandb_args arguments. The command was working few days ago. I really don't understand.

[haileyschoelkopf]

Hi! What is the current commit that you're working off of?

I'm unable to replicate this but perhaps there's some difference in our setups.

[TheRealM4rtin]

I'm working on the commit 9e6e240229429d2214bc281bed7a4e288f5169a1

[TheRealM4rtin]

The issue was related to the PATH environment variable prioritizing a different lm_eval installation (~/.local/bin). Thanks for the incredibly quick response @haileyschoelkopf