georgian-io / Multimodal-Toolkit

Multimodal model for text and tabular data with HuggingFace transformers as building block for text data
https://multimodal-toolkit.readthedocs.io
Apache License 2.0
591 stars 85 forks source link

XLNet doesn't support the "weighted_feature_sum_on_transformer_cat_and_numerical_feats" combining module #46

Closed Aymen-lab closed 1 year ago

Aymen-lab commented 1 year ago
Dear team, 

I am doing several experiments with the toolkit. Thank you very much for the good work. While trying XLNet with the combination module "weighted_feature_sum_on_transformer_cat_and_numerical_feats". I got the following error:

230             self.act_func = create_act(self.mlp_act)
231             self.layer_norm = nn.LayerNorm(self.text_out_dim)

--> 232 self.final_dropout = nn.Dropout(tabular_config.hidden_dropout_prob) 233 self.final_out_dim = self.text_out_dim 234 AttributeError: 'TabularConfig' object has no attribute 'hidden_dropout_prob'

I think in model initialization this configuration is missing. Please update the code. tabular_config.hidden_dropout_prob = hf_model_config.hidden_dropout_prob