search

automl / TabPFN

Official implementation of the TabPFN paper (https://arxiv.org/abs/2207.01848) and the tabpfn package.
http://priorlabs.ai
Apache License 2.0
1.22k stars 109 forks source link

Meaning of mix_activations #79

Open amueller opened 10 months ago

amueller commented 10 months ago

My understanding of mix_activations is whether to mix activation functions within a batch when sampling from the MLP prior. Now there's

        if not (('mix_activations' in hyperparameters) and hyperparameters['mix_activations']):
            s = hyperparameters['prior_mlp_activations']()
            hyperparameters['prior_mlp_activations'] = lambda: s

which seems to support that. Now there's this comment:

config["mix_activations"] = False # False heisst eig True

which seems to imply True and False are switched. Can you confirm that the comment is out of date, or am I missing something?