In adapter-fine-tuning, why don't fix original params?

[zihaolucky]

Hi guys,

I have a glance at the run_classifier.py code and didn't see the code for fixing original transformer parameters, so it's full fine-tune setting, and why? Thanks~

[ghost]

@zihaolucky The restriction to training only parameters in the adapters, layer-norm, and head is made in the optimizer, using collections:

tvars = []
for collection in ["adapters", "layer_norm", "head"]:
  tvars += tf.get_collection(collection)

[zihaolucky]

@neil-houlsby Get it. Thank you!