Inquery about the dimension of input and output

[luli191]

Hello, after reading your library, I would say it is a great job for uncertainty calibration, but I was wondering if the model is limited that in bayes regression model, input and output must have the same size, but sometimes you know this is kind of a big limitation for dataset, maybe sometimes input dimension is like 10 and target dimension is like 4

[o-laurent]

Hi @luli191,

Thank you for your message and your interest in the library. I may need some more information to answer precisely to your question.

It seems that it is already possible to select the output dimension of your regression problem. For instance, there is an output_dim argument for the regression routine. Moreover, the basic Bayesian MLP that we provide also includes an argument for this (num_outputs). Could you provide a more precise pointer towards the class that does not support this feature - and that you are interested in - so that I can help?

[luli191]

Hi@o-laurent ,

Thank you for your reply.

I think that even if the dimension of output_dim can be selected, it still requires that the input and output must be of the same dimension, because in my regression attempt, the _forward function in bayes_linear.py will return F.inear with an error as long as input and output is not the same size：RuntimeError: linear(): input and weight.T shapes cannot be multiplied (3x10 and 4x64).

I'm now interested in how to train a bayesian regression model to predict target values don't have the same size with input, like x=[1,4,7,,9,10], y=[3e-1,5e-2].

And it seems that there is not a prediction function for bayes regression, I didn't find that, I would be great appreciate if you could tell me where to find.

Thank you.

[o-laurent]

Hi again @luli191,

Could you send me the code defining the model that you are using? Could you also tell me what is your batch size and the dimensions of your data (x and y)?

[luli191]

Hi @o-laurent, sure, here is the model I am using.

# model
def optim_regression(
    model: nn.Module,
    learning_rate: float = 5e-4,
):
    optimizer = optim.Adam(
        model.parameters(),
        lr=learning_rate,
        weight_decay=0,
    )
    return optimizer

model = bayesian_mlp(
    in_features=10,
    num_outputs=4,
    hidden_dims=[64, 64],
)

loss = ELBOLoss(
    model=model,
    inner_loss=nn.MSELoss(),
    kl_weight=1 / 50000,
    num_samples=3,
)

routine = RegressionRoutine(
    probabilistic=False,
    output_dim=4,
    model=model,
    loss=loss,
    optim_recipe=optim_regression(model),
)

batch_size is 32, x is a one-dimensional tensor with 10 columns, y is a one-dimensional tensor with 4 columns, like x=[1,2,3,4,5,6,7,8,9,10], y=[1,2,3,4].

If you could give me your email address, I think I can send you the complete code file with dataset.

Thank you.

[alafage]

Hi @luli191,

Thank you for the code snippet; it allowed us to discover that the ELBOLoss was not correctly handled within the RegressionRoutine. The package's next release will fix this issue (it should be released tomorrow). In the meantime, you could always try install the fixed version using the following command:

pip install https://github.com/ENSTA-U2IS/torch-uncertainty/archive/dev.zip

Let us know if you need any other help.

[luli191]

Hi@alafage,

Also, I believe it's necessary to encapsulate a predict method in regression routine. This is particularly important for uncertainty quantification problems, as being able to directly see the predicted values is very intuitive for understanding the model's output.

Your project is a valuable contribution for uncertainty calibration.

Thank you!

[alafage]

The RegressionRoutine already has a method for prediction; it uses the forward method of the model. You should be able to get predictions from your model using:

inputs = ...  #some inputs
routine = RegressionRoutine(...)

predictions = routine(inputs)
# or
predictions = routine.forward(inputs)

In addition, you can access the model using routine.model.

I hope that helps.

[luli191]

@alafage

Thank you for your reminder. I had previously looked at the RegressionRoutine, but I overlooked the forward method, which led me to think there wasn't a method provided for obtaining predictions. Perhaps adding a tutorial for Bayesian regression would be a good idea？

[alafage]

Thanks for the suggestion. I am putting it on the todo list :)