Closed neildhir closed 5 years ago
mikkokotila
commented
5 years ago You don't have to pass params['something'] as argument to every parameter in a Keras model. You can just pass something instead. Or if you want to have it visible in your Talos experiment results, then just have a single value for that parameter in your params dictionary. In your case that would be {... class_weight: [someweight] ...}.
Closing this as resolved. Feel free to open new issue if anything.
neildhir
commented
5 years ago @mikkokotila thanks for your quick response. I believe I am misunderstanding how talos treats parameters.
You said:
You don't have to pass params['something'] as argument to every parameter in a Keras model. You can just pass something instead.
I should clarify that I need to pass this parameter dynamically as this will change depending on the dataset I am using, and I do not want to hardcode this array in the main model itself, but pass it as a (static) array which does not get optimised. But because the talos syntax looks as so (in my case):
t = ta.Scan(vstack([X_train,X_test]),
asarray(y_train+y_test).reshape(-1, 1),
model=char_cnn_model_talos,
disable_progress_bar=True,
params=opt_params,
val_split=.1)the only parameters I can pass to my model, because of talos, is the dictionary opt_params. Consequently my something needs to be a value with a key in the opt_params dictionary. This bring us to your second point:
Or if you want to have it visible in your Talos experiment results, then just have a single value for that parameter in your params dictionary. In your case that would be {... class_weight: [someweight] ...}.
This would indeed be good but if I have a multiclass or binary-class classification problem then the class-weight array will be an array an not a single number. And then, I believe, talos will explicitly try to find the optimal value (for these values) w.r.t. to the objective. This is what I am trying to avoid. I simply want to pass that array without talos "finding out" about it as it were. Here is an example of my parameter dict which at present does not work:
opt_params ={
'optimizer': ['adam'],
'loss': ['binary_crossentropy'],
'conv_activation':['elu'],
'dense_activation':['elu'],
'last_activation': ['sigmoid'],
# Stationary parameters, i.e. do not get optimised
'class_weight': {0:0.5, 1:1.2}
'max_sentence_length':[max_sentence_length]
}Currently this results in the following error:
~/anaconda3/lib/python3.6/site-packages/talos/scan/scan_run.py in scan_run(self)
7
8 from .scan_prepare import scan_prepare
----> 9 self = scan_prepare(self)
10
11 # initiate the progress bar
~/anaconda3/lib/python3.6/site-packages/talos/scan/scan_prepare.py in scan_prepare(self)
30 round_limit=self.round_limit,
31 time_limit=self.time_limit,
---> 32 boolean_limit=self.boolean_limit
33 )
34
~/anaconda3/lib/python3.6/site-packages/talos/parameters/ParamSpace.py in __init__(self, params, param_keys, random_method, fraction_limit, round_limit, time_limit, boolean_limit)
33
34 # create list of list from the params dictionary
---> 35 self._params_temp = [list(self.p[key]) for key in self.param_keys]
36
37 # establish max dimensions
~/anaconda3/lib/python3.6/site-packages/talos/parameters/ParamSpace.py in <listcomp>(.0)
33
34 # create list of list from the params dictionary
---> 35 self._params_temp = [list(self.p[key]) for key in self.param_keys]
36
37 # establish max dimensions
KeyError: 'class_weight'And for clarity, here is a snippet from my model which I am trying to optimise:
# > Compile
model.compile(loss=params['loss'],
optimizer=params['optimizer'],
metrics=['accuracy'])
# > Fit model
out = model.fit(X_train,
y_train,
validation_data=(X_test, y_test),
verbose=0, # Set to zero if using live plotting of losses
class_weight=params['class_weight'],
batch_size=params['batch_size'],
epochs=params['epochs'])
return out, modelTo start, there is an error in:
opt_params ={
'optimizer': ['adam'],
'loss': ['binary_crossentropy'],
'conv_activation':['elu'],
'dense_activation':['elu'],
'last_activation': ['sigmoid'],
# Stationary parameters, i.e. do not get optimised
'class_weight': {0:0.5, 1:1.2}
'max_sentence_length':[max_sentence_length]
}As you can see, a comma is missing after row with class_weight.
Also, dictionary is not allowed as an input format, so that would lead to more errors.
When you say "I don't want this to get optimized", let me try and clarify. If you passlast_activation': ['sigmoid'] in to your params dictionary, then it is exactly the same as directly setting activation="sigmoid" on the output layer of your Keras model.
For 'class_weight': {0:0.5, 1:1.2} specifically, if you have some condition inside the model that is supposed to decide which value to pick (0 or 1 key), then you have to handle this as a conditional statement in the input model itself. As long as you don't include params['class_weight'] in the input model as a parameter anywhere, Talos will not do anything about it.
neildhir
commented
5 years ago Thank you for your swift reply.
As you can see, a comma is missing after row with class_weight.
Quite. This is my sort of "ideal" opt dictionary. This is not something I am using, more as a demonstration of what I would like.
Also, dictionary is not allowed as an input format, so that would lead to more errors.
Yupp fully aware of that, just trying to show you what I am getting at.
When you say "I don't want this to get optimized", let me try and clarify. If you passlast_activation': ['sigmoid'] in to your params dictionary, then it is exactly the same as directly setting activation="sigmoid" on the output layer of your Keras model.
Yupp all is well in that department too. The problem arises if I try to pass say last_activation': ['sigmoid','ramp' ] then talos would try both settings. I have a setting where I would like to pass a an array-like parameter but not optimise it.
For 'class_weight': {0:0.5, 1:1.2} specifically, if you have some condition inside the model that is supposed to decide which value to pick (0 or 1 key), then you have to handle this as a conditional statement in the input model itself. As long as you don't include params['class_weight'] in the input model as a parameter anywhere, Talos will not do anything about it.
This is what I was trying to get at. Right now I have hardcoded the class_weight as so:
out = model.fit(X_train,
y_train,
validation_data=(X_test, y_test),
verbose=2, # Set to zero if using live plotting of losses
class_weight={0: 0.7770370370370371, 1: 1.4024064171122994},
# Monitor the loss with early stopping
callbacks=[EarlyStopping(patience=5, min_delta=0.0001)],
batch_size=params['batch_size'],
epochs=params['epochs'])and you are saying that with talos this is really the only option?
This is a very simple issue.
My model is defined as so:
with the fit part, inside the above definition, looking like so:
Here is my problem: under the Talos syntax, I need to define my model as above, but this also means I cannot (?) pass arguments to my model, arguments that I do not want optimised. I.e. I would like to pass the
class_weighttomodel.fit()but I do not know how to do that since the formatting in talos does not allow me model arguments beyondX_train,y_train,X_test,y_test,params-- but if I placeclass_weightinsideparamsthen implicitly this means that I wantclass_weightoptimised, which I do not. I just want to pass it as a static parameter tomodel.fit().How would one achieve this?