search

zjunlp / EasyEdit

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.
https://zjunlp.github.io/project/KnowEdit
MIT License
1.74k stars 210 forks source link

Cannot run IKE on counterfact #258

Closed AmitRozner closed 3 months ago

AmitRozner commented 3 months ago

Hi, I am trying to run IKE on counterfact dataset using the following command: python run_knowedit_llama2.py \ --editing_method=IKE \ --hparams_dir=../hparams/IKE/llama-7b \ --data_dir=./data \ --datatype='counterfact' Since train_data_path is not set it is breaking at: train_ds = KnowEditDataset(args.train_data_path) The error is due to train_data_path being None: TypeError: expected str, bytes or os.PathLike object, not NoneType

Any advice? Thanks

XeeKee commented 3 months ago

The IKE method requires providing some cases as examples. Therefore, when running it, you need to include the train_data_path parameter, similar to the MEND and SERAC methods.

zxlzr commented 3 months ago

Hi, have you solved your issue?

AmitRozner commented 3 months ago

I tried with KnowEdit/benchmark/wiki_counterfact/train_cf.json how much GPU memory is required for llama-7b? I am trying with A100 40gb and getting OOM. Also, what is the file for ZsRE? I tried zsre/zsre_mend_train_10000.jsonbut getting an error: "prompt": record["prompt"] if "prompt" in record else record["text"], KeyError: 'text'

XeeKee commented 3 months ago

run_knowedit_llama2.py is prepared for KnowEdit, so your dataset should be selected in KnowEdit. You can download it from https://huggingface.co/datasets/zjunlp/KnowEdit. If you encounter an out-of-memory error, you can reduce the value of k in the hparams.

zxlzr commented 3 months ago

Hi, have you solved your issue?

AmitRozner commented 3 months ago

I lowered K and was able to run it partially. Need to get an 80GB GPU to run everything.