Unrecognized argument for --apply_chat_template

[tginart]

(xenv-0017-phi3) (xenv-0017) aginart@ip-10-1-123-117:~/dev$ lm_eval --model hf --model_args pretrained=EleutherAI/gpt-j-6B,trust_remote_code=True --tasks hellaswag --device cuda:0 --batch_size=8 --apply_chat_template 2024-07-23:01:51:15,897 INFO [utils.py:148] Note: NumExpr detected 48 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8. 2024-07-23:01:51:15,897 INFO [utils.py:161] NumExpr defaulting to 8 threads. 2024-07-23:01:51:16,268 INFO [config.py:58] PyTorch version 2.2.0 available. No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda-12.1' usage: lm_eval [-h] [--model MODEL] [--tasks task1,task2] [--model_args MODEL_ARGS] [--num_fewshot N] [--batch_size auto|auto:N|N] [--max_batch_size N] [--device DEVICE] [--output_path DIR|DIR/file.json] [--limit N|0<N<1] [--use_cache DIR] [--decontamination_ngrams_path DECONTAMINATION_NGRAMS_PATH] [--check_integrity] [--write_out] [--log_samples] [--show_config] [--include_path DIR] [--gen_kwargs GEN_KWARGS] [--verbosity CRITICAL|ERROR|WARNING|INFO|DEBUG] lm_eval: error: unrecognized arguments: --apply_chat_template

• Clone master
• run basic command with apply_chat_tempalte
• Somehow arg is unrecognized?

[drishyakarki]

hi did you solve this?