search

automl / PFNs

Our maintained PFN repository. Come here to train SOTA PFNs.
Apache License 2.0
51 stars 10 forks source link

activation choice bug in simple MLP prior #5

Closed doronator closed 7 months ago

doronator commented 8 months ago

the design clearly is meant to allow choosing amongst various activations: https://github.com/automl/PFNs/blob/main/pfns/priors/simple_mlp.py#L29

But the simple MLP ends up always using tanh in the forward pass: https://github.com/automl/PFNs/blob/main/pfns/priors/simple_mlp.py#L57

the fix is simply to replace: x = torch.tanh(x) with: x = self.activation(x)

SamuelGabriel commented 7 months ago

good catch! should be fixed now.

SamuelGabriel commented 7 months ago

just tested, is fixed now :)