search

mingkaid / rl-prompt

Accompanying repo for the RLPrompt paper
MIT License
286 stars 52 forks source link

About the prepended special character \u0120. #35

Closed guozix closed 11 months ago

guozix commented 11 months ago

Thanks for releasing the code!

I have some questions about the prepended special character \u0120 in verbalizers:

  1. What's the difference between classifying in ['\u0120terrible', '\u0120great'] and in ['terrible', 'great']? Are '\u0120great' and 'great' recognized as two different words in the verbalizing stage?
  2. Why verbalizers of some of the tasks, e.g. agnews, yahoo, do not have these prepended \u0120?
mingkaid commented 11 months ago

Thank you for your interest! Please refer to this issue for the explanation. Basically, this is GPT-2's way of representing "space". Let me know if you have any other question. I am closing this issue for now because it's a clarification question.

guozix commented 11 months ago

Thanks for your patient reply! greatly appreciated! I understand that the involvement of \u0120 is naturally caused by the construction of the BPE tokenizer based on pre-train data.

I have another three questions:

  1. Is setting the verbalizers with "\u0120-prepended" words a conventional design or just an empirical design for RLPrompt?
  2. Is using "\u0120-prepended" verbalizers aims for the result of, take sentiment classification as example, "\<sentence> It was\<space>great" instead of "\<sentence> It wasgreat" ?
  3. Have you tried to use "non-\u0120-prepended" verbalizers? Can it achieve comparable results to current design of verbalizers?
MM-IR commented 11 months ago
  1. A conventional Design.
  2. The former one.
  3. Yes, it depends, and in most cases, the results are competitive.