autonomio / talos

Hyperparameter Experiments with TensorFlow and Keras
https://autonom.io
MIT License
1.62k stars 268 forks source link

talos for a model with custom layer #534

Closed ruhollah2 closed 3 years ago

ruhollah2 commented 3 years ago

2) Include the output of:

talos.__version__ 1.0.0

3) Explain clearly what you expect to happen

I have the same problem as here.

A description of what you tried . In the following, I tried to access the best model via a dictionary (as suggested here by the poster of the same issue):

First, I ran Analyze with the scan_object to find the round w/ best result using rounds2high. Say we store the best model number into model_no:

analyze_object_rh = talos.Analyze(scan_object)
model_no = analyze_object_rh.rounds2high('out.reg.vae_func_correlation_coefficient')# get the round with the best result

Then I tried the following, but it failed:

ruh_best_model = talos.utils.best_model.activate_model({scan_object,{'myCsutomLayer': myCsutomLayer}} , model_no) due to sytaxterr:

SyntaxError: positional argument follows keyword argument

So, then I tired another workaround suggested by the talos developer here:

from talos.utils.recover_best_model import recover_best_model
results, models = recover_best_model( x_train = x_train
                                     , y_train = y_train
                                     , x_val= x_val
                                     , y_val= y_val 
                                     , experiment_log = 'ruh_custom_vae_talos/012621170617.csv'
                                     , input_model = ruh_func_customvae_maker
                                     , n_models = 5
                                     , task = 'continuous'
                                    )

It also did fail with the following err:

KeyError: 'val_acc'

(after all, how should I know which model is the best to choose the corresponding .csv file in the ?! Here I randomly chose 012621170617.csv just to see if this workaround works, but it failed anyways)

Well, here is my model.compile() (it is a multi-task learning model)

 my_model_custom_vae.compile( loss ={'decoder.Model': 'mse' , 'out.reg.vae':'mse'}
                                ,loss_weights = {'out.reg.vae': input_params['loss_wieghts_layer_outReg'], 'decoder.Model':input_params['loss_wieghts_layer_decoderModel']} 
                                ,optimizer = myoptimizer_adam
                                , metrics= {'out.reg.vae':[func_correlation_coefficient], 'decoder.Model':[func_correlation_coefficient]}
                               )
mikkokotila commented 3 years ago

KeyError: 'val_acc'

This is now fixed in #546.

Merging this with #421 for the part on better handling of custom layers.