automl / TabPFN

Official implementation of the TabPFN paper (https://arxiv.org/abs/2207.01848) and the tabpfn package.
http://priorlabs.ai
Apache License 2.0
1.22k stars 109 forks source link

Can we do Class weighting in TabPFN? #48

Closed leosv123 closed 1 year ago

leosv123 commented 1 year ago

How to give weightage the Class while training using TabPFNClassifier?

leosv123 commented 1 year ago

Making Changes to this function with custom weighting?

def JointBCELossWithLogits(output, target):
    # output shape: (S, B, NS) with NS = Number of sequences
    # target shape: (S, B, SL)
    # Loss = -log(mean_NS(prod_SL(p(target_SL, output_NS))))
    # Here at the moment NS = SL
    output = output.unsqueeze(-1).repeat(1, 1, 1, target.shape[-1]) # (S, B, NS, SL)
    output = output.permute(2, 0, 1, 3) # (NS, S, B, SL)
    print(target.shape, output.shape)
    loss = (target * torch.sigmoid(output)) + ((1-target) * (1-torch.sigmoid(output)))
    loss = loss.prod(-1)
    loss = loss.mean(0)
    loss = -torch.log(loss)
    loss = loss.mean()
    return loss
nsarang commented 11 months ago

@noahho Is this implemented?