ml5js / ml5-examples

A collection of ml5.js examples
https://examples.ml5js.org
MIT License
626 stars 450 forks source link

ImageClassification example doesn't work with tensorflow.js model generated with google autoML #263

Closed gerchicov-bp closed 4 years ago

gerchicov-bp commented 4 years ago

Tried to assign a new tensorflow.js model generated by google to your image classification example. Code changed:

    classifier = ml5.imageClassifier('./tfjs/model.json');
    img = loadImage('./images/abyssinian.jpg'); //image is any image of cat/dog/plant

Example: https://github.com/ml5js/ml5-examples/tree/release/p5js/ImageClassification/ImageClassification model: tfjs.zip

The code is almost unchanged. So does ml5.js has bugs or google generates the wrong model?

sketch.js:22 Error: layer: Improper config format: {"node":[{"name":"image","op":"Placeholder","attr":{"shape":{"shape":{"dim":[{"size":"1"},{"size":"224"},{"size":"224"},{"size":"3"}]}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/feature_extractor/Mean/reduction_indices","op":"Const","attr":{"value":{"tensor":{"dtype":"DTINT32","tensorShape":{"dim":[{"size":"2"}]}}},"dtype":{"type":"DTINT32"}}},{"name":"mnas_v4_a_140/output/fc/tf_layer/kernel","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"1280"},{"size":"3"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140/output/fc/tf_layer/bias","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"3"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"ExpandDims/dim","op":"Const","attr":{"value":{"tensor":{"dtype":"DTINT32","tensorShape":{}}},"dtype":{"type":"DTINT32"}}},{"name":"mnas_v4_a_140_1/feature_network/stem/conv/Conv2D_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"3"},{"size":"3"},{"size":"3"},{"size":"48"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/stem/conv/Conv2D_bn_offset","op":"Const","attr":{"dtype":{"type":"DTFLOAT"},"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"48"}]}}}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_0/op_0/depthwise_0/depthwise_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"3"},{"size":"3"},{"size":"48"},{"size":"1"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_0/op_0/depthwise_0/depthwise_bn_offset","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"48"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_0/op_0/project_0/Conv2D_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"1"},{"size":"1"},{"size":"48"},{"size":"24"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_0/op_0/project_0/Conv2D_bn_offset","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"24"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_1/op_0/expand_0/Conv2D_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"1"},{"size":"1"},{"size":"24"},{"size":"72"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/cell_14/op_0/depthwise_0/depthwise_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"5"},{"size":"5"},{"size":"1632"},{"size":"1"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_1/op_0/expand_0/Conv2D_bn_offset","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"72"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_1/op_0/depthwise_0/depthwise_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"3"},{"size":"3"},{"size":"72"},{"size":"1"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_1/op_0/depthwise_0/depthwise_bn_offset","op":"Const","attr":{"dtype":{"type":"DTFLOAT"},"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"72"}]}}}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_1/op_0/project_0/Conv2D_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"1"},{"size":"1"},{"size":"72"},{"size":"24"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/cell_14/op_0/depthwise_0/depthwise_bn_offset","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"1632"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_1/op_0/project_0/Conv2D_bn_offset","op":"Const","attr":{"dtype":{"type":"DTFLOAT"},"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"24"}]}}}}},{"name":"mnas_v4_a_140_1/feature_network/cell_2/op_0/expand_0/Conv2D_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"1"},{"size":"1"},{"size":"24"},{"size":"72"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/cell_2/op_0/expand_0/Conv2D_bn_offset","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"72"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/cell_2/op_0/depthwise_0/depthwise_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"3"},{"size":"3"},{"size":"72"},{"size":"1"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/cell_2/op_0/depthwise_0/depthwise_bn_offset","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShaptf-layers.esm.js:17) at E (tf-layers.esm.js:17) at un (tf-layers.esm.js:17) at tf-layers.esm.js:17 at tf-layers.esm.js:17 at Object.next (tf-layers.esm.js:17) at o (tf-layers.esm.js:17) gotResult @ sketch.js:22 (anonymous) @ callcallback.js:14 Promise.catch (async) e.default @ callcallback.js:13 (anonymous) @ index.js:208 x @ runtime.js:62 (anonymous) @ runtime.js:296 t.<computed> @ runtime.js:114 i @ asyncToGenerator.js:17 (anonymous) @ asyncToGenerator.js:15 e @ _export.js:36 (anonymous) @ asyncToGenerator.js:14 (anonymous) @ index.js:208 setup @ sketch.js:14 (anonymous) @ p5.min.js:3 _runIfPreloadsAreDone @ p5.min.js:3 _decrementPreload @ p5.min.js:3 i.onload @ p5.min.js:3 load (async) x.loadImage @ p5.min.js:3 (anonymous) @ p5.min.js:3 preload @ sketch.js:9 (anonymous) @ p5.min.js:3 m @ p5.min.js:3 n @ p5.min.js:3 load (async) 21.../core/main @ p5.min.js:3 l @ p5.min.js:3 (anonymous) @ p5.min.js:3 12../color/color_conversion @ p5.min.js:3 l @ p5.min.js:3 o @ p5.min.js:3 (anonymous) @ p5.min.js:3 (anonymous) @ p5.min.js:3 (anonymous) @ p5.min.js:3 tf-layers.esm.js:17 Uncaught (in promise) Error: layer: Improper config format: {"node":[{"name":"image","op":"Placeholder","attr":{"shape":{"shape":{"dim":[{"size":"1"},{"size":"224"},{"size":"224"},{"size":"3"}]}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/feature_extractor/Mean/reduction_indices","op":"Const","attr":{"value":{"tensor":{"dtype":"DTINT32","tensorShape":{"dim":[{"size":"2"}]}}},"dtype":{"type":"DTINT32"}}},{"name":"mnas_v4_a_140/output/fc/tf_layer/kernel","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"1280"},{"size":"3"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140/output/fc/tf_layer/bias","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"3"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"ExpandDims/dim","op":"Const","attr":{"value":{"tensor":{"dtype":"DTINT32","tensorShape":{}}},"dtype":{"type":"DTINT32"}}},{"name":"mnas_v4_a_140_1/feature_network/stem/conv/Conv2D_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"3"},{"size":"3"},{"size":"3"},{"size":"48"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/stem/conv/Conv2D_bn_offset","op":"Const","attr":{"dtype":{"type":"DTFLOAT"},"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"48"}]}}}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_0/op_0/depthwise_0/depthwise_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"3"},{"size":"3"},{"size":"48"},{"size":"1"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_0/op_0/depthwise_0/depthwise_bn_offset","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"48"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_0/op_0/project_0/Conv2D_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"1"},{"size":"1"},{"size":"48"},{"size":"24"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_0/op_0/project_0/Conv2D_bn_offset","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"24"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_1/op_0/expand_0/Conv2D_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"1"},{"size":"1"},{"size":"24"},{"size":"72"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/cell_14/op_0/depthwise_0/depthwise_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"5"},{"size":"5"},{"size":"1632"},{"size":"1"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_1/op_0/expand_0/Conv2D_bn_offset","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"72"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_1/op_0/depthwise_0/depthwise_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"3"},{"size":"3"},{"size":"72"},{"size":"1"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_1/op_0/depthwise_0/depthwise_bn_offset","op":"Const","attr":{"dtype":{"type":"DTFLOAT"},"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"72"}]}}}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_1/op_0/project_0/Conv2D_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"1"},{"size":"1"},{"size":"72"},{"size":"24"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/cell_14/op_0/depthwise_0/depthwise_bn_offset","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"1632"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/lead_cell_1/op_0/project_0/Conv2D_bn_offset","op":"Const","attr":{"dtype":{"type":"DTFLOAT"},"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"24"}]}}}}},{"name":"mnas_v4_a_140_1/feature_network/cell_2/op_0/expand_0/Conv2D_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"1"},{"size":"1"},{"size":"24"},{"size":"72"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/cell_2/op_0/expand_0/Conv2D_bn_offset","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"72"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/cell_2/op_0/depthwise_0/depthwise_weights","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShape":{"dim":[{"size":"3"},{"size":"3"},{"size":"72"},{"size":"1"}]}}},"dtype":{"type":"DTFLOAT"}}},{"name":"mnas_v4_a_140_1/feature_network/cell_2/op_0/depthwise_0/depthwise_bn_offset","op":"Const","attr":{"value":{"tensor":{"dtype":"DTFLOAT","tensorShaptf-layers.esm.js:17) at E (tf-layers.esm.js:17) at un (tf-layers.esm.js:17) at tf-layers.esm.js:17 at tf-layers.esm.js:17 at Object.next (tf-layers.esm.js:17) at o (tf-layers.esm.js:17) e @ tf-layers.esm.js:17 E @ tf-layers.esm.js:17 un @ tf-layers.esm.js:17 (anonymous) @ tf-layers.esm.js:17 (anonymous) @ tf-layers.esm.js:17 (anonymous) @ tf-layers.esm.js:17 o @ tf-layers.esm.js:17

ab-dev-github commented 4 years ago

I guess I am facing similar issue, I have created a issue - https://github.com/ml5js/ml5-examples/issues/276 - Is this what you are facing too?

Were you able to resolve it?

joeyklee commented 4 years ago

Hi @gerchicov-bp - As far as I'm aware, ml5.js has not integrated support for Google's AutoML models. So far the only external model training that is supported is via Google's Teaching Machine where we only support models trained in the image classifier and the sound classifier.

To @ab-dev-github - custom trained posenet models in Teachable machine are not supported in ml5.js for now.

I will close this for now. In case there are further questions, feel free to reopen! 🙏