Multitargets regression "None"prediction issue

[altar31]

When I tried to perform a multitargets regression on tabular data the model prediction output is "None" as shown in the screenshot below.

Please find the requirements.txt for the .venv used : requirements.txt

Finally, this is the minimal example :

import random
import pandas as pd
import numpy as np
import pickle
import torch
from pytorch_widedeep.preprocessing import TabPreprocessor
from pytorch_widedeep.models import TabMlp, WideDeep
from pytorch_widedeep.losses_multitarget import MultiTargetRegressionLoss
from pytorch_widedeep import Trainer

# generate data
data = {
    "X1": [random.uniform(200, 250) for _ in range(1000)],
    "X2": [random.uniform(100, 200) for _ in range(1000)],
    "X3": [random.uniform(0, 15000) for _ in range(1000)],
    "Y1": [random.uniform(100, 2000) for _ in range(1000)],
    "Y2": [random.uniform(40, 100) for _ in range(1000)],
}
df = pd.DataFrame(data)

test_data = {
    "X1": [random.uniform(220, 250) for _ in range(1000)],
    "X2": [random.uniform(156, 200) for _ in range(1000)],
    "X3": [random.uniform(0, 15000) for _ in range(1000)],
}
df_test = pd.DataFrame(test_data)

# set the target
target = df[["Y1", "Y2"]].values.astype(np.float32)
# Tabular preprocessor
tab_preprocessor = TabPreprocessor(continuous_cols=["X1", "X2", "X3"])
X_tab = tab_preprocessor.fit_transform(df).astype(np.float32)
# Model architecture
tab_mlp = TabMlp(
    column_idx=tab_preprocessor.column_idx,
    continuous_cols=tab_preprocessor.continuous_cols,
    mlp_hidden_dims=[64, 32],
)
model = WideDeep(deeptabular=tab_mlp, pred_dim=2)
print("The model architecture:\n", model)

# Multitargets loss
loss = MultiTargetRegressionLoss(weights=[0.5, 0.5], reduction="mean")
# Set the trainer
trainer = Trainer(model, objective="multitarget", custom_loss_function=loss)
# Fit the model
trainer.fit(
    X_tab=X_tab,
    target=target,
    n_epochs=20,
    batch_size=16,
)

# save the model
torch.save(model.state_dict(), "multi_regression_torch.pt")
# save preprocessors
with open("multi_regression_.pkl", "wb") as dp:
    pickle.dump(tab_preprocessor, dp)
print("Training completed and model saved.")

# Load the saved preprocessor
with open(f"multi_regression_.pkl", "rb") as tp:
    tab_preprocessor_new = pickle.load(tp)

# Load the trained model for inference on test data
X_test = tab_preprocessor_new.transform(df_test).astype(np.float32)
new_model = WideDeep(deeptabular=tab_mlp, pred_dim=2)
new_model.load_state_dict(torch.load(f"models/multi_regression_torch.pt"))
new_model.eval()

# Use the trained model for prediction
trainer_new = Trainer(model, objective="multitarget", custom_loss_function=loss)
preds = trainer_new.predict(X_tab=X_test, batch_size=64)
print("Predictions array:\n", preds)

Maybe the issue came from the "poor" training loss value (in this case, as I train it on a very small dataset for illustration purpose). However, in the real setting, i have tried to fine-tune the model hyperparameters and I trained the model on a bigger dataset too. But it still leads me to poor loss value around 800...

So the issue is it related to the training dataset, model architecture, the training loop or the inference step ?

[altar31]

My apologies, they is a small issue in the code above.

Please find the correct minimal example to try :

import random
import pandas as pd
import numpy as np
import pickle
import torch
from pytorch_widedeep.preprocessing import TabPreprocessor
from pytorch_widedeep.models import TabMlp, WideDeep
from pytorch_widedeep.losses_multitarget import MultiTargetRegressionLoss
from pytorch_widedeep import Trainer

# generate data
data = {
    "X1": [random.uniform(200, 250) for _ in range(1000)],
    "X2": [random.uniform(100, 200) for _ in range(1000)],
    "X3": [random.uniform(0, 15000) for _ in range(1000)],
    "Y1": [random.uniform(100, 2000) for _ in range(1000)],
    "Y2": [random.uniform(40, 100) for _ in range(1000)],
}
df = pd.DataFrame(data)

test_data = {
    "X1": [random.uniform(220, 250) for _ in range(1000)],
    "X2": [random.uniform(156, 200) for _ in range(1000)],
    "X3": [random.uniform(0, 15000) for _ in range(1000)],
}
df_test = pd.DataFrame(test_data)

# set the target
target = df[["Y1", "Y2"]].values.astype(np.float32)
# Tabular preprocessor
tab_preprocessor = TabPreprocessor(continuous_cols=["X1", "X2", "X3"])
X_tab = tab_preprocessor.fit_transform(df).astype(np.float32)
# Model architecture
tab_mlp = TabMlp(
    column_idx=tab_preprocessor.column_idx,
    continuous_cols=tab_preprocessor.continuous_cols,
    mlp_hidden_dims=[64, 32],
)
model = WideDeep(deeptabular=tab_mlp, pred_dim=2)
print("The model architecture:\n", model)

# Multitargets loss
loss = MultiTargetRegressionLoss(weights=[0.5, 0.5], reduction="mean")
# Set the trainer
trainer = Trainer(model, objective="multitarget", custom_loss_function=loss)
# Fit the model
trainer.fit(
    X_tab=X_tab,
    target=target,
    n_epochs=40,
    batch_size=16,
)

# save the model
torch.save(model.state_dict(), "multi_regression_torch.pt")

# save preprocessors
with open("multi_regression_.pkl", "wb") as dp:
    pickle.dump(tab_preprocessor, dp)
print("Training completed and model saved.")

# Load the saved preprocessor
with open("multi_regression_.pkl", "rb") as tp:
    tab_preprocessor_new = pickle.load(tp)

# Load the trained model for inference on test data
X_test = tab_preprocessor_new.transform(df_test).astype(np.float32)
new_model = WideDeep(deeptabular=tab_mlp, pred_dim=2)
new_model.load_state_dict(torch.load("multi_regression_torch.pt"))
new_model.eval()

# Use the trained model for prediction
trainer_new = Trainer(model, objective="multitarget", custom_loss_function=loss)
preds = trainer_new.predict(X_tab=X_test, batch_size=16)
print("Predictions array:\n", preds)

[jrzaurin]

This is a bug at inference time. it will be fixed asap :)

thanks for opening the issue!

[jrzaurin]

Hi @altar31

Could you do this please:

pip install git+https://github.com/jrzaurin/pytorch-widedeep.git@ffm

(or just clone the repo and branch out to ffm)

and then run your example again.

Should work.

I will soon be merging this branch which will fix this issue

[altar31]

Hi @jrzaurin

Perfect, it seems to work now as you can see below

Thanks a lot ! 👍

I will try it on a real world example asap ! 🎉