search

nyu-mll / jiant-v1-legacy

The jiant toolkit for general-purpose text understanding models
MIT License
21 stars 9 forks source link

[CLOSED] Update __init__.py so that MiniBERTas can be probed #1100

Closed jeswan closed 4 years ago

jeswan commented 4 years ago

Issue by YianZhang Sunday Jun 28, 2020 at 17:48 GMT Originally opened as https://github.com/nyu-mll/jiant/pull/1100

Update from the inductive-bias-pretraining team (Sam, Alex, Haokun, Haau-Sing, Yian). Four new roberta model names are added to transformer_input_module_to_tokenizer_name so that our new models can be used as input_modules in jiant.

YianZhang included the following code: https://github.com/nyu-mll/jiant/pull/1100/commits

jeswan commented 4 years ago

Comment by pep8speaks Sunday Jun 28, 2020 at 17:48 GMT

Hello @YianZhang! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

Line 10:101: E501 line too long (105 > 100 characters)

You can repair most issues by installing black and running: black -l 100 ./*. If you contribute often, have a look at the 'Contributing' section of the README for instructions on doing this automatically.

jeswan commented 4 years ago

Comment by sleepinyourhat Sunday Jun 28, 2020 at 18:46 GMT

@YianZhang Thanks!

jeswan commented 4 years ago

Comment by YianZhang Sunday Jun 28, 2020 at 19:34 GMT

@sleepinyourhat

This update is for the somewhat hacky solution we agreed on on slack: In the tutorial we will require the users to download the checkpoint themselves, rename it and put it under the root directory. I did not look into auto-download feature, but I can have a try. Do you think this is necessary?

jeswan commented 4 years ago

Comment by sleepinyourhat Sunday Jun 28, 2020 at 19:44 GMT

Ah, got it. If I understand correctly, this won't be an advertised/core model in jiant, but there will be a tutorial on an external page explaining how to use it. That sounds reasonable to me, since this is just a pointer to another checkpoint file for the same model—we can pretty much guarantee that no future PR will break support for these models without also breaking support for RoBERTa.

@pyeres: Any objection to something like this? Anything we should keep in mind?

jeswan commented 4 years ago

Comment by YianZhang Sunday Jun 28, 2020 at 20:05 GMT

@sleepinyourhat Yes, you understand it correctly.

@sleepinyourhat @pyeres I checked the code again and I don't think it will be crazy to include them as an advertised/core model in jiant, though it might take a little longer for debugging. Another option is to start officially supporting these models in jiant2.0 if you find it worthwhile.

jeswan commented 4 years ago

Comment by YianZhang Monday Jun 29, 2020 at 22:30 GMT

Please ignore this PR. I will make a new one where MiniBERTas can be used just like other input_modules.