search

kundajelab / deeplift

Public facing deeplift repo
MIT License
818 stars 162 forks source link

Importing models from JSON #65

Open TommyJW opened 5 years ago

TommyJW commented 5 years ago

I noticed while doing an export / import of a keras model it doesn't parse Keras's model_to_json() output anymore.

The fix I've been using is to remove the 'Layers' child of the first 'Config' parent.

JSON output {"class_name": "Sequential", "config": {"name": "sequential_1", "layers": [{"class_name": "Conv1D", "config": {"name": "conv1d_1", "trainable": true, "batch_input_shape": [null, 1948, 4], "dtype": "float32", "filters": 64, "kernel_size": [3], "strides": [1], "padding": "valid", "data_format": "channels_last", "dilation_rate": [1], "activation": "relu", "use_bias": true, "kernel_initializer": {"class_name": "VarianceScaling", "config": {"scale": 1.0, "mode": "fan_avg", "distribution": "uniform", "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}}, {"class_name": "Flatten", "config": {"name": "flatten_1", "trainable": true, "data_format": "channels_last"}}, {"class_name": "Dense", "config": {"name": "dense_1", "trainable": true, "units": 100, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "VarianceScaling", "config": {"scale": 1.0, "mode": "fan_avg", "distribution": "uniform", "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}}, {"class_name": "Dropout", "config": {"name": "dropout_1", "trainable": true, "rate": 0.5, "noise_shape": null, "seed": null}}, {"class_name": "Dense", "config": {"name": "dense_2", "trainable": true, "units": 1, "activation": "linear", "use_bias": true, "kernel_initializer": {"class_name": "VarianceScaling", "config": {"scale": 1.0, "mode": "fan_avg", "distribution": "uniform", "seed": null}}, "bias_initializer": {"class_name": "Zeros", "config": {}}, "kernel_regularizer": null, "bias_regularizer": null, "activity_regularizer": null, "kernel_constraint": null, "bias_constraint": null}}, {"class_name": "Activation", "config": {"name": "activation_1", "trainable": true, "activation": "sigmoid"}}]}, "keras_version": "2.2.4", "backend": "tensorflow"}

Error: `--------------------------------------------------------------------------- TypeError Traceback (most recent call last)

in 4 h5_file=keras_model_weights, 5 json_file=keras_model_json, ----> 6 nonlinear_mxts_mode=NonlinearMxtsMode.DeepLIFT_GenomicsDefault) 7 ~/.local/share/virtualenvs/SeniorProject-AFH1tCxG/lib/python3.6/site-packages/deeplift/conversion/kerasapi_conversion.py in convert_model_from_saved_files(h5_file, json_file, yaml_file, **kwargs) 361 for layer_config in layer_configs: 362 --> 363 layer_name = layer_config["config"]["name"] 364 assert layer_name in model_weights,\ 365 ("Layer "+layer_name+" is in the layer names but not in the " TypeError: string indices must be integers ` Accepted JSON `{"class_name":"Sequential","config":[{"class_name":"Conv1D","config":{"name":"conv1d_1","trainable":true,"batch_input_shape":[null,1948,4],"dtype":"float32","filters":64,"kernel_size":[3],"strides":[1],"padding":"valid","data_format":"channels_last","dilation_rate":[1],"activation":"relu","use_bias":true,"kernel_initializer":{"class_name":"VarianceScaling","config":{"scale":1.0,"mode":"fan_avg","distribution":"uniform","seed":null}},"bias_initializer":{"class_name":"Zeros","config":{}},"kernel_regularizer":null,"bias_regularizer":null,"activity_regularizer":null,"kernel_constraint":null,"bias_constraint":null}},{"class_name":"Flatten","config":{"name":"flatten_1","trainable":true,"data_format":"channels_last"}},{"class_name":"Dense","config":{"name":"dense_1","trainable":true,"units":100,"activation":"linear","use_bias":true,"kernel_initializer":{"class_name":"VarianceScaling","config":{"scale":1.0,"mode":"fan_avg","distribution":"uniform","seed":null}},"bias_initializer":{"class_name":"Zeros","config":{}},"kernel_regularizer":null,"bias_regularizer":null,"activity_regularizer":null,"kernel_constraint":null,"bias_constraint":null}},{"class_name":"Dropout","config":{"name":"dropout_1","trainable":true,"rate":0.5,"noise_shape":null,"seed":null}},{"class_name":"Dense","config":{"name":"dense_2","trainable":true,"units":1,"activation":"linear","use_bias":true,"kernel_initializer":{"class_name":"VarianceScaling","config":{"scale":1.0,"mode":"fan_avg","distribution":"uniform","seed":null}},"bias_initializer":{"class_name":"Zeros","config":{}},"kernel_regularizer":null,"bias_regularizer":null,"activity_regularizer":null,"kernel_constraint":null,"bias_constraint":null}},{"class_name":"Activation","config":{"name":"activation_1","trainable":true,"activation":"sigmoid"}}],"keras_version":"2.2.4","backend":"tensorflow"}` Attached are the files, changed to .txt. for uploading, the h5 wont upload due to size constraints [JSON conversion Issue.txt](https://github.com/kundajelab/deeplift/files/2469508/JSON.conversion.Issue.txt) [JSONkerasbase_model.txt](https://github.com/kundajelab/deeplift/files/2469509/JSONkerasbase_model.txt) [JSONkerasbase_modelAdjusted.txt](https://github.com/kundajelab/deeplift/files/2469510/JSONkerasbase_modelAdjusted.txt)
TommyJW commented 5 years ago

simple fix for now:

#Keras outputs its JSON with an extra 'Layers" parent, DeepLift can't parse
#this object, so this function makes the conversion
def convertKerasJSONtoDeepLIFT(kerasJSON_str):
    jsonData = json.loads(kerasJSON_str)
    layersData = jsonData["config"]["layers"]
    jsonData["config"] = layersData
    return json.dumps(jsonData)