search

YangRui2015 / RiC

Code for the ICML 2024 paper "Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment"
38 stars 3 forks source link

Questions on dataset processing #1

Closed TingchenFu closed 5 months ago

TingchenFu commented 5 months ago

In ric/utils.py line 223, I guess the sample["input_ids"] = tokenizer.encode(sample["text"]) should be sample["input_ids"] = tokenizer.encode(sample["prompt"]) instead.

YangRui2015 commented 5 months ago

Hi,

During the evaluation phase, we use prompts with desired scores as input. In line 154 of evaluation.py, we changed the 'input_ids' to 'prompt_with_score_ids'. While the process is correct, there is redundant processing. I will work to improve the readability soon.

valid_dataset = valid_dataset.remove_columns('input_ids') valid_dataset=valid_dataset.rename_column('prompt_with_score_ids', 'input_ids')

TingchenFu commented 5 months ago

Thanks for your quick reply! Indeed the logic is correct and I just missed the details.