nyu-mll / nlu-test-sets

Analysis of NLU test sets with IRT
10 stars 8 forks source link

question about the training step in jiant #2

Closed Xiaoqianhou closed 3 years ago

Xiaoqianhou commented 3 years ago

Hello~ We wanted to replicate what you have done in jiant (following this instruction https://github.com/nyu-mll/nlu-test-sets/tree/main/jiant_scripts) but we found that the 'Training a model' step is too time-consuming for us. So we wanted to skip this step (we can do the tokenizing , we just want to skip training step) and use the pretrained models we download in the 'Downloading pretrained models' step to generate responses. But after trying for a few days we still don't get any success. So we want to ask that is it possible to do the things we want? If possible, could you please kindly give us some instructions? about where we have to change in your code? We'd be really thankful if you can give us some advice! Thank you in advance

sleepinyourhat commented 3 years ago

The ~18 pretrained models there are things like BERT encoders, which extract representations for text, but don't produce any output.

I'm sorry to say, I don't believe we don't have the full task-trained models from the paper available. They're huge, and the overhead of uploading/storing them is larger than just rerunning fine-tuning.

Xiaoqianhou commented 3 years ago

oh okay then~ Thank you for your answer!

在2021-07-23 22:36:14,Sam @.***写道:

The ~18 pretrained models there are things like BERT encoders, which extract representations for text, but don't produce any output.

I'm sorry to say, I don't believe we don't have the full task-trained models from the paper available. They're huge, and the overhead of uploading/storing them is larger than just rerunning fine-tuning.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.