search

jordiclive / ControlPrefixes

Apache License 2.0
90 stars 11 forks source link

Question: control prefixes without task prefix #5

Closed base-y closed 2 years ago

base-y commented 2 years ago

HI, I see control prefixes being used along with task prefixes in the code. Can I ran control prefixes without task prefix and will that affect any part of the code? Also, wondering the thought behind using both hard prompt task prefix (translate graph to english - as mentioned in the paper) and soft-prompt task prefix.

jordiclive commented 2 years ago

Yes, you can run without a task prefix, and just have Control Prefixes.

The soft-prompt task prefix consists of additional key-value pairs at every layer so it has a lot more parameters acting at every layer, and therefore is more expressive.

T5 is a model we chose as the underlying frozen model. Performance is usually better with prompts such as "Translate graph to english". You are right this small textual prompt is not very important in our case, as we are learning lots of parameters to 'prompt' the language model.