IBM / Dromedary

Dromedary: towards helpful, ethical and reliable LLMs.
GNU General Public License v3.0
1.13k stars 87 forks source link

What a human annotator should do? #4

Closed yhyu13 closed 1 year ago

yhyu13 commented 1 year ago

May be a stupid question, I am not sure what prompts messages I should edit as a human annotator.

https://github.com/IBM/Dromedary/tree/main/prompts

It seems I should give customized hint and instruction in the tgrt_self_instruct_question_generation_prompt.txt file?

What about the seed tasks? Can they be customized to specific areas?

I am not sure I understand what I should touch and What I should not touch

Edward-Sun commented 1 year ago

Hi,

Thank you for your interest in our work. To adapt our method to your own application, I would suggest you modify two prompts:

  1. the main self-alignment prompt: such as add or remove rules for your application
  2. the synthetic questions types: It would also be a good idea to add new question types that you want to cover in the rule engraving process, for example, "Questions that require medical context" or "Questions that require medical knowledge"