[Speed issue]: How would you recommend running EasyEdit faster

[domenicrosati]

Hey there,

I am running EasyEdit for LLama2 and it is very very very slow (only doing 500 edits in 10 days on 4 vl100 GPUs). I am loading a new process (python main.py edit_sample 0, python main.py edit_sample 1 etc.) for each edit because i need to evaluate each edit on the original model (i.e. the original ROME-style counterfact setting)

Is there a faster way to do this? I saw there is a squential_edit=False option now does this mean I can keep things in the same proccess and run through edits in a single process?

[pengzju]

Hello, I've never had this problem before. But I just thought you were editing the model with KN, my suggestion is that you can adjust the value of p, try 0.9 or 0.01 which is faster, by testing a single instance. In addition, this is not Easyedit's problem, KN needs to collect neuron in proportion, we can't optimize this piece of content

[domenicrosati]

@pengzju this is for all methods.

It isn't EasyEdit's problem since for each edit I have to run a seperate python process. I am asking if there is a way to avoid starting a new process requiring loading the original model into memory for every edit that people have identified.

[pengzju]

I understand that you mean to enable the editor in a lazy way, and only load the original model when the editor is actually used for editing. It sounds like some kind of collaborative editing, but Easyedit doesn’t have a related interface at the moment. The suggestion I can give is to separate ‘self.model’ and change from in-class attributes to global variables, which requires you to change the code of editor. I don't plan to schedule this, and please understand.