search

XinyuanWangCS / PromptAgent

This is the official repo for "PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization". PromptAgent is a novel automatic prompt optimization method that autonomously crafts prompts equivalent in quality to those handcrafted by experts, i.e., expert-level prompts.
https://arxiv.org/abs/2310.16427
Apache License 2.0
140 stars 14 forks source link

关于摘要的prompt #6

Closed sumingfirst closed 3 months ago

sumingfirst commented 3 months ago

庆祝大佬拿下ICLR2024，可以询问一下，如果我想用他来做文本摘要的prompt的工作，是不是按照新加task部分设置数据集和相关评价指标就可以啦,祝大佬多发paper

XinyuanWangCS commented 3 months ago

Thanks.

对。你需要在src/task里面加入你写的.py文件。task类继承base_task，transform_format, clean_response等处理格式的函数可能需要根据你的数据调整；cal_correct函数一般按直接比较相等，cal_metric是在eval set上计算的reward，你可能根据需要实现关于文本摘要的metric，e.g. ROUGE score。如果你的文本很长，建议使用小的batch_size, e.g. 3。