keras-team / autokeras

AutoML library for deep learning
http://autokeras.com/
Apache License 2.0
9.14k stars 1.4k forks source link

[Question] Using complex custom modalities with AutoKeras #1651

Open neel04 opened 2 years ago

neel04 commented 2 years ago

Firstly, thanks a lot to the authors and maintainers for providing such a versatile and dynamic tool! šŸš€

I had a dataset with an image, on which I would like to regress 2 different floats/per image. I believe such a task comes under the Multi-Task category of the documentation.

In such a case where the output heads are actually the same, does something like this work:-

# Initialize the multi with multiple inputs and outputs.
model = ak.AutoModel(
    inputs=[ak.ImageInput()],
    outputs=[
        ak.RegressionHead(metrics=["mae"]),
        ak.RegressionHead(metrics=["mae"])), # 2 regression heads, for 2 continuous variables
    ],
    overwrite=True,
    max_trials=2,

I assume that its computing something like how torch does behind the scenes

criterion_1 = MAE()
criterion_2 = MAE()
loss = criterion_1(regression_head_1, gt_1) + criterion_2(regression_head_2, gt_2)
loss.backward()
......

Where it forms 2 different criteria and is backpropagated together.

So would passing my data with such an architecture be enough for AutoKeras to backprop? if there are modifications to be made, could you kindly provide me with the base files where I can start from?

Many Many Thanks šŸ¤—

neel04 commented 2 years ago

@haifeng-jin I know you might be busy with a lot of work, but if you could just take out 2 minutes and provide some guidance; it would be very, very welcome!