search

tensorspace-team / tensorspace

Neural network 3D visualization framework, build interactive and intuitive model in browsers, support pre-trained deep learning models from TensorFlow, Keras, TensorFlow.js
https://tensorspace.org
Apache License 2.0
5.04k stars 444 forks source link

[Question]: Loading Keras models into appropriate TSP layers automatically #219

Closed Marviel closed 5 years ago

Marviel commented 5 years ago

Hey, awesome project! Forgive me if this question is answered elsewhere.

I noticed that the InceptionV3 example seems to manually specify the javascript definition of the Tensorspace models, even though I think that the same information could be inferred given the .json file within that example.

Am I wrong about this? I know there is an issue with some backends (tf.js?) not providing enough information for automated extraction and encoding into Tensorspace models, but was under the impression that Keras based models contained the necessary data.

Have I messed up in my understanding of the .json keras model definitions?

Marviel commented 5 years ago

Oh, this may just be solved by #213, when it closes :)

syt123450 commented 5 years ago

@Marviel Thanks for your suggestions!

As you said, tfjs model can provide some information for TensorSpace layers, and it is possible to get these information automatically, and I am designing the auto-loader to automatically create TensorSpace model with given pre-trained model.

Theoretically, it is possible to get all information from tfjs layer model or graph model. As TensorSpace mainly used to visualize a pre-trained model, some layers will not take effect in model prediction process, such as, batchnorm, dropout, and so on, and these layers will not be visualized in TensorSpace. From my perspective, auto-loader needs to infer a sub-set of pre-trained model's topo structure, it is a little bit complex, and may take time to design and implement it.

I am actively developing this feature, it will come in future version of TensorSpace~