search

georgian-io / Multimodal-Toolkit

Multimodal model for text and tabular data with HuggingFace transformers as building block for text data
https://multimodal-toolkit.readthedocs.io
Apache License 2.0
591 stars 85 forks source link

Batch Normalization is done even with False BN parameter #54

Closed speter00 closed 1 year ago

speter00 commented 1 year ago

In the model/tabular_combiner.py script, everytime MLP is initialized the bn parameter is set to True, ignoring the value of self.numerical_bn (the parameter normally responsible for determining whether batch normalization is done or not). This causes most feature combination methods to end up using Batch Normalization even if the numerical_bn parameter was set to False in TabularConfig. An easy fix is to simply do bn=self.numerical_bn each time MLP is called, so that the parameter is accounted for instead of using True everytime.

akashsaravanan-georgian commented 1 year ago

Thank you @speter00, this looks like a duplicate of #53. We'll fix in the next release but feel free to send a PR and we can merge it in!

speter00 commented 1 year ago

It is not a duplicate. This is a separate issue. #53 was about a misleading parameter name used in an example. This is about a part of the code ignoring the value of this same parameter, and instead using a fixed value.

akashsaravanan-georgian commented 1 year ago

Ah whoops, apologies I misread it then. But feel free to send a PR :)