Where is the code of "TensorNetwork for Machine Learning" for MNIST/FASHION MNIST?

[jiawen11]

Hi,

I could not find the code of "TensorNetwork for Machine Learning" for MNIST/FASHION MNIST as example for Machine Learning. May I know where I can find the code in as the paper shows?

Thanks!

[mganahl]

Thanks for asking! We have removed the code in release 0.1.1. If you download release 0.1.0 you can find it in experiments/MPS_classifier.

[jiawen11]

Thanks for asking! We have removed the code in release 0.1.1. If you download release 0.1.0 you can find it in experiments/MPS_classifier.

Thank you for your quick reply, Martin!

One more question, I could not find any performance results (throughput, e.g., images per second) of employing TensorNetwork (TN) on ML with MNIST in neither the paper nor online resources. Did you compare the performance difference between TN on ML and mainstrea ML framework (e.g., TensorFlow) for image classification with CNN on MNIST/FASHION MNIST? If so, what is the performance difference (e.g., 100 images/second on V100 NVIDIA GPU) of training/inference to achieve the top-1 accuracy (e.g., 99%)?

I would appreciate if you have any performance evaluation results of employing TN on ML with MNIST.

Thanks!

[mganahl]

Unfortunately I don't know if there are such results. Maybe @stavros11 can help, he was the main contributor to this project.

[stavros11]

Thank you for asking (and thanks to Martin for mentioning me).

As Martin said the code was removed from this repository in a previous release, however it is still available in a seperate repository. As noted in the README, there were breaking changes in TensorNetwork after we did this project and therefore the code will most likely not work with the latest version of TensorNetwork.

Regarding performance, all the results in the paper were about the Matrix Product State (MPS) classifier implemented in TensorNetwork and there was no comparison with other machine learning methods, such as CNNs. From my small experience, neural networks can obtain the same and better accuracies faster than this MPS method. This should be expected as NNs are the standard approach in machine learning. However exact performance also depends on the implementation. For example, the simplest way to train the MPS classifier is with "brute force" gradient descent using the automatic differentation provided out-of-the-box by all machine learning libraries. This was the method used in our paper. In the original work by Stoudenmire and Schwab they used a physics-inspired sweeping method which may be faster if implemented properly.

For the sake of giving some numbers, I wrote a quick Tensorflow/Keras implementation of the MPS classifier (in the same spirit used in our paper) and a CNN from a Keras tutorial. You can check the code in the Colab if you like. Without any parameter tuning, both models got to ~95% test accuracy on MNIST with the CNN training at 15ms / step, and the MPS at 45ms / step (1 step = batch of 128 images). I used Colab's free GPU as I do not have access to any other GPU service. Note that this is just a quick implementation and is not very well documented (sorry!).

[chaserileyroberts]

Thanks Stavros! I'll leave this issue open since we've been asked about this before and someone will want likely ask again.

[jiawen11]

Hi Stavros,

Thank you so much for your detailed reply! It's very helpful!

And thanks to Martin to invite Stavros to answer this question.

Thanks!

[chaserileyroberts]

Let's keep it open for others with the same question.

[SamuelMarks]

Great to see this issue, I was about to add a new one with the same question.

With that in mind, you mention that this hasn't been updated for the latest version of TensorNetwork.

Are you planning to update it? - If you want me to update it, can you give a checklist of what would be necessary?

Thanks

[8bitmp3]

@Thenerdstation @stavros11 @mganahl @sleichen Hi and HNY 🥳 Thanks for working on this awesome library and keeping the basic classifier example in https://github.com/Thenerdstation/TensorNetsForML and in the Colab. Can you help find some ML examples in JAX with TN? I think @Thenerdstation mentioned at NeurIPS 2019 and the X/TN meetup (November 2019?) that you've used TF and JAX. I understand that you could write a custom TNLayer class with a backend=jax in JAX/Flax and replace one of the Dense layers with the most params, for example, but I wanted to see some examples. Thank you!

[cnmozzie]

I also keep part of the codes here: https://github.com/cnmozzie/MPS_classifier (made some changes so that it can be compitable with tf2)