How to train model by changing the number of layers in the model?

facebookresearch / SpanBERT

Code for using and evaluating SpanBERT.

Other

884 stars 174 forks source link

How to train model by changing the number of layers in the model? #92

Open RAJA-PARIKSHAT opened 1 year ago

RAJA-PARIKSHAT commented 1 year ago

I want to train spanBert from scratch, for contextual span embedding. The architecture is quite large and I think only three layers will suffice for my task. Please help me with how can I reduce the size of the architecture and train it on the objective of spanbert. Thanks