search

poloclub / cnn-explainer

Learning Convolutional Neural Networks with Interactive Visualization.
https://poloclub.github.io/cnn-explainer/
MIT License
8.1k stars 1.22k forks source link

Possible to support loading and introspecting via ONNX.js? #12

Closed EricCousineau-TRI closed 4 years ago

EricCousineau-TRI commented 4 years ago

Relates #2 and #8: Might it be possible to support loading / introspecting (a subset of) models loaded via ONNX.js? https://github.com/Microsoft/onnxjs

Curious because we would like to visualize our own PyTorch model / networks.

I'm assuming that this may be closed as out-of-scope, which is totally fine! Just wanna put it out there ;)

EricCousineau-TRI commented 4 years ago

An add'l link regarding PyTorch <-> TensorFlow via ONNX, on top of the links provided via https://github.com/poloclub/cnn-explainer/issues/8#issuecomment-624624004 https://medium.com/@santhoshkolloju/convert-your-pytorch-models-to-tensorflow-with-onnx-84c3bdd8d722

xiaohk commented 4 years ago

Hey, thanks for you suggestion!

Yeah we have considered onnx.js vs tensorflow.js when we started the project. From our experience, tensorflow.js was more straightforward at that time.

It is definitely possible to support loading models with onnx.js. In cnn-tf.js we converted the loaded model into a JavaScript array of objects, and all visualization code are using this array. Therefore, after constructing this array using onnx.js all other code should work as well :)

However, it might be a little bit tricky to modify the CNN Explainer code to inspect your model (assuming not tiny-vgg 😅) out of the box, as the visualization layout is especially designed for tiny-vgg in the current version.

I will close this issue for now, let me know if you have more questions :)