search

dreamquark-ai / tabnet

PyTorch implementation of TabNet paper : https://arxiv.org/pdf/1908.07442.pdf
https://dreamquark-ai.github.io/tabnet/
MIT License
2.55k stars 470 forks source link

Minimal working example for TabNetRegressor/Classifier #519

Closed SurtaiHan closed 9 months ago

SurtaiHan commented 9 months ago

Hi, I'm trying to make a minimal working example to understand how to use TabNet.

Currently, with the features as the identity matrix of size 3, and the target values as a vector [3,4,5], the final fitted model seems to predict negative values.

If the current behavior is a bug, please provide the steps to reproduce. The full code to reproduce the issue is shown below, along with the current output.

Expected behavior I expect it to correctly produce [3,4,5] on the given input because it seems like this should be learnable even with a linear regression. Although I'm not sure if this expectation is unrealistic for DNNs.

Screenshots image

image

Other relevant information: poetry version:
python version: Operating System: Additional tools:

Additional context

SurtaiHan commented 9 months ago

I'm also a bit confused why loss=0 makes sense alongside a non-zero mse.

SurtaiHan commented 9 months ago

Seems like when batch_size is greater than the number of datapoints, loss is 0. When I set batch size to 2, it works for the classifier case.

Optimox commented 9 months ago

During training, drop_last=True for the dataloader, so if your entire dataset is smaller than your batch size then the model is never trained at all. loss=0 is because it is instanciated with the value 0.

SurtaiHan commented 9 months ago

that worked, thanks!