search

keras-team / keras-hub

Pretrained model hub for Keras 3
Apache License 2.0
797 stars 243 forks source link

XLMRobertaBackbone has no Kernel_initializer defined #863

Closed soma2000-lang closed 1 year ago

soma2000-lang commented 1 year ago

@abheesht17 @mattdangerw I think this needs to be fixed

shivance commented 1 year ago

Hi, if you take a look at XLMRobertaClassifier , you'll find that we are reusing roberta_kernel_initializer in XLMRoberta.

The reason is simple, XLMRoberta inherits from Roberta (try to look around in XLM Roberta Backbone). Furthermore, just to be clear, it's just a function that can be implemented seperately for XLMRoberta, and initializes kernel weight with random normal distribution.

soma2000-lang commented 1 year ago

@shivance Thanks for clarification.!

abheesht17 commented 1 year ago

@shivance, thanks for helping out here! :)