Closed nguyentthong closed 3 years ago
Can you share with us the code you use to conduct experiments in your paper?
We have used the exact code in this repository to conduct all of our experiments. Instructions on how to use the code to reproduce our results can be found in the README file. Let me know if you have any further questions on how to use the code or on how to reproduce a specific experiment :)
Oh, I just realized that you were asking for a different paper - sorry! The text generation paper is currently under review; we'll release the source code once review is complete.
By when can I expect the source code? :)
On Thu, Jun 10, 2021, 23:04 Timo Schick @.***> wrote:
Oh, I just realized that you were asking for a different paper - sorry! The text generation paper is currently under review; we'll release the source code once review is complete.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/timoschick/pet/issues/34#issuecomment-858822842, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASA3VFYM5FGJWCSBLS34FNLTSDZTLANCNFSM45YFCKWA .
I can't make any promises because I'm still waiting for the results of the review and I'm really busy right now, but I guess "somewhen next month" is a reasonable estimate :)
Great! Thanks. But is text generation with PET really that different from
text classification?
Like, for classification, maybe review sentiment analysis you use patterns
like this:
[Review here] I found it <MASK>.
Here the verbalizer will be {'+1': ['good'], '-1': ['bad']} the <MASK>
is
predicted as 'good' or 'bad'?
Wouldn't text generation be similar, like if you want to complete a
sentence, your pattern would be even simpler:
[Incomplete sentence] <MASK>
The verbalizer would be every word in the English language each mapped to a
label, and then it would pick the correct word from the verbalizer for the
<MASK>
token and the sentence will be completed...right?
Great! Thanks. But is text generation with PET really that different from text classification? Like, for classification, maybe review sentiment analysis you use patterns like this:
[Review here] I found it <MASK>.
Here the verbalizer will be {'+1': ['good'], '-1': ['bad']} the<MASK>
is predicted as 'good' or 'bad'? Wouldn't text generation be similar, like if you want to complete a sentence, your pattern would be even simpler:[Incomplete sentence] <MASK>
The verbalizer would be every word in the English language each mapped to a label, and then it would pick the correct word from the verbalizer for the<MASK>
token and the sentence will be completed...right?
@BleepLogger I am not sure whether you are aware of this, but Schick & Schütze talk about the differences between implementing PET for classification vs. text generation in this new paper, to which the code is missing (but the paper is there): "Few Short Text Generation with Pattern Exploiting"
It is probably best to just read it yourself (it's only 6 pages), but let me highlight some misconceptions about why it's almost the same:
like if you want to complete a sentence, your pattern would be even simpler:
[Incomplete sentence] <MASK>
Text generation includes much more than just filling slots with one word. E.g., that's not how you summarize a text.
The verbalizer would be every word in the English language each mapped to a label
Every word in the English language? There is no finite amount of words. You would have to go with tokens, which are limited. But predicting one token is not what text generation is about. Also, there is no verbalizer in PET for text generation.
Hope you found that helpful! :) I encourage you to look at the paper!
@npbeck I found that VERY helpful! Thank you so much, I just finished reading the paper. Still waiting for the code to be released though...
@timoschick Been waiting long... Please release the code for text generation with PET...
@thongnguyen050999 @BleepLogger @npbeck : You can now find the code for GenPET at this feature branch. It also contains a couple of new features that are not part of the arXiv preprint. Unfortunately, I didn't have time yet to update the Readme section to explain how to use PET for text generation and I'll only be able to do so in the first week of August (I'm on vacation next week), but feel free to experiment with the code already. Looking at the changes in cli.py
should hopefully give you an idea of how to use GenPET. Note that this feature is still experimental, so let me know if anything does not work as expected.
I'm closing this issue for now. If you encounter any specific problems with the implementation of GenPET, please create a new issue for it.
Dear @timoschick
I have read and found you paper "Few-Shot Text Generation with Pattern-Exploiting Training" really interesting. Can you share with us the code you use to conduct experiments in your paper?