search

OpenBMB / ModelCenter

Efficient, Low-Resource, Distributed transformer implementation based on BMTrain
https://modelcenter.readthedocs.io
Apache License 2.0
233 stars 28 forks source link

[FEATURE] Add custom config parameters in **from_pretrained** #9

Closed xcjthu closed 2 years ago

xcjthu commented 2 years ago

Is your feature request related to a problem? Please describe.

Sometimes, we need to customize some config parameters and remain others unchanged. For example, during the finetuning, I want to set the dropout rate as 0.1. Then I have to write a new config. It would be nice if I can use T5.from_pretrained("t5-large", dropout_p=0.1) to set the dropout rate.

Achazwl commented 2 years ago

added in #13 for both model.from_pretrained(path, **kwargs) and config.from_pretrained(path, **kwargs). Please note that currently model.from_pretrained(path, config=) and model.from_pretrained(path, **kwargs) cannot be used at the same time as model.from_pretrained(path, config=, **kwargs).