maxpumperla / hyperas

Keras + Hyperopt: A very simple wrapper for convenient hyperparameter optimization
http://maxpumperla.com/hyperas/
MIT License
2.17k stars 317 forks source link

How do I use it given a network with multiple inputs? #81

Open Miail opened 7 years ago

Miail commented 7 years ago

Can this optimiser be used in keras models with multiple inputs.. I use fit_generator as the data cannot be in ram, so tried to follow the example given, and implemented this for multiple inputs.

I added a MVCE for reproducibility.

https://pastebin.com/zH62fHBt

But i am getting error message:

Traceback (most recent call last):
  File "keras_cnn_phoneme_original_fit_generator_hyperas.py", line 211, in <module>
    trials=Trials())
  File "/usr/local/lib/python2.7/dist-packages/hyperas/optim.py", line 43, in minimize
    notebook_name=notebook_name, verbose=verbose)
  File "/usr/local/lib/python2.7/dist-packages/hyperas/optim.py", line 63, in base_minimizer
    model_str = get_hyperopt_model_string(model, data,functions,notebook_name, verbose, stack)
  File "/usr/local/lib/python2.7/dist-packages/hyperas/optim.py", line 130, in get_hyperopt_model_string
    imports = extract_imports(cleaned_source, verbose)
  File "/usr/local/lib/python2.7/dist-packages/hyperas/utils.py", line 44, in extract_imports
    import_parser.visit(tree)
  File "/usr/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/usr/lib/python2.7/ast.py", line 249, in generic_visit
    self.visit(item)
  File "/usr/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/usr/local/lib/python2.7/dist-packages/hyperas/utils.py", line 14, in visit_Import
    if (self._import_asnames(node.names)!=''):
  File "/usr/local/lib/python2.7/dist-packages/hyperas/utils.py", line 36, in _import_asnames
    return ''.join(asname)
TypeError: sequence item 0: expected string, NoneType found

Something you could elaborate on?

maxpumperla commented 7 years ago

It seems you're using what we call "intermediate functions", would you mind checking out this example and tell me if it works for you? In principle, there's nothing fundamentally in your example that shouldn't work:

https://github.com/maxpumperla/hyperas/blob/master/examples/use_intermediate_functions.py

ChandrahasJR commented 7 years ago

Hi @maxpumperla, does x_train, y_train, x_test, y_test all have to be tensors ?

If i have multiple input tensors to my network, can i wrap all of them in a dict ?

Example : return {'Input_1_train' : ip1_train_tensor, 'Input_2_train' : ip2_train_tensor}, { 'Output_train' : op_train_tensor}, {'Input_1_test' : ip1_test_tensor, 'Input_2_test' : ip2_test_tensor}, { 'Output_test' : op_test_tensor}

amw5g commented 5 years ago

I, too, would like to use multiple inputs (single output) to a model. This is because different parts of my data need different processing inside the model (e.g., batchnormalization or embedding). So I currently use named inputs like so: 'X_input': X_train, 'foo_input': foo_train, 'bar_input': bar_train In reality, there are more than 3 inputs. and I have similarly named validation inputs for validating So before I go about wrapping them all up in a dictionary, and testing/debugging, is there an agreed upon method to pass multiple inputs to hyperopts?

amw5g commented 5 years ago

I just submitted a PR that showcases an example of using multiple inputs within a jupyter notebook. It builds off the simple example. https://github.com/maxpumperla/hyperas/pull/196