search

rabeehk / compacter

126 stars 15 forks source link

Compacter Issue #4

Closed jmohta closed 2 years ago

jmohta commented 2 years ago

Hi Rabeeh,

Thanks for releasing the code. I was trying to run compacters but I am facing some issue running it with the current config values. In the config (https://github.com/rabeehk/compacter/blob/main/seq2seq/configs/compacter.json#L44) file parameter shared_phm_rule is set to true. In the hypercomplex/layers.py file there is a if condition (https://github.com/rabeehk/compacter/blob/main/seq2seq/hypercomplex/layers.py#L69) for initializing phm_rule. But, as shared_phm_rule is set to true self.phm_rule is not initialized and this will cause problems in the forward pass? Thanks for your help.

rabeehk commented 2 years ago

Hi @jmohta thanks for the question. This is initialized inside the model, please see https://github.com/rabeehk/compacter/blob/b210eef13f64ff6441186ee5a1cbf031b5918b94/seq2seq/third_party/models/t5/modeling_t5.py#L1552

Best regards Rabeeh

jmohta commented 2 years ago

thanks! makes sense.