First of all, thanks for supporting multiple token verbalizers. This feature is critical to my scenario. My verbalizers typically have 4-6 tokens.
As mentioned in https://github.com/timoschick/pet#pet-with-multiple-masks, using multi-mask PET is very very slow at evaluation step because 1) max eval batch size has to be 1; 2) Multiple forward passes needs to be run for each eval step.
My questions are:
1) Is there any plan to address the evaluation step speed issue with some new design?
2) Is there any alternative way to speed up the evaluation step?
Unfortunately, we currently do not have any plans to address the evaluation step speed issue. However, as the last step of PET is to train a regular sequence classifier on data annotated by the individual models, the final model should be fast (it's just the intermediate evaluations and the generation of soft-labeled data that takes very long).
As an alternative, you may try using a generative language model with GenPET, but our current implementation of GenPET only supports PEGASUS, so you'd have to modify some of the code to make it work for other generative models like T5.
First of all, thanks for supporting multiple token verbalizers. This feature is critical to my scenario. My verbalizers typically have 4-6 tokens.
As mentioned in https://github.com/timoschick/pet#pet-with-multiple-masks, using multi-mask PET is very very slow at evaluation step because 1) max eval batch size has to be 1; 2) Multiple forward passes needs to be run for each eval step.
My questions are: 1) Is there any plan to address the evaluation step speed issue with some new design? 2) Is there any alternative way to speed up the evaluation step?
Thanks.