Is SEARC's training loss same with paper?

[Luoyang144]

Hi, I'm tring to read the training code of SEARC, I wonder where is the loss and it seems not same with origin paper?

Here is loss I find in editable_model.py:

def _edit_loss_fn(config, pred, targ, **kwargs):
    if 'minigpt4' in config.model_name.lower() or 'blip' in self.config.model_name.lower():
        return masked_log_probs(config, pred, targ, exact_match=self.config.exact_match, shift=True, **kwargs)
    elif 't5' in config.model_class.lower():
        return masked_log_probs(config, pred, targ,)
    elif 'gpt' in config.model_class.lower():
        return masked_log_probs(config, pred, targ, shift=True, **kwargs)
    elif 'llama' in config.model_class.lower():
        return masked_log_probs(config, pred, targ, shift=True, **kwargs)
    elif 'internlm' in config.model_name.lower():
        return masked_log_probs(config, pred, targ, shift=True)
    elif 'chatglm' in config.model_name.lower():
        return masked_log_probs(config, pred, targ, shift=True)
    elif 'qwen' in config.model_name.lower():
        return masked_log_probs(config, pred, targ, shift=True)
    elif 'mistral' in config.model_name.lower():
        return masked_log_probs(config, pred, targ, shift=True)
    else:
        return masked_log_probs(config, pred, targ,)

self.edit_loss_fn = _edit_loss_fn
self.loc_loss_fn = masked_log_probs

It seems not same with follow loss which used in SERAC?

[XeeKee]

The training loss of the counterfact model is NLL. The masked_log_probs is just an entry point, and the actual execution is at https://github.com/zjunlp/EasyEdit/blob/38c5c34d614646db14a45f59790434bc2f520c1b/easyeditor/trainer/losses.py#L55, where we calculate the NLL.

[zxlzr]

Hi, do you have any further questions?

[Luoyang144]

@XeeKee Hi, thanks for reply, which function is the execution of scope classifier's loss?

[XeeKee]

Here, we referenced the source code from SERAC. The classifier and the small model are trained together.

[zxlzr]

Hi, do you have any further questions?