Clustering / Unsupervised Learning

[shiffman]

I am working on some examples to cluster data sets. Here are some algorithms I imagine eventually having in this library:

• kmeans
• principal component analysis (PCA)
• TSNe

I committed a kmeans example (uses random vectors) as a start.

https://github.com/ITPNYU/p5-deeplearn-js/tree/master/examples/clustering/kmeans

Some next steps are:

• [ ] Is the algorithm correct? (I whipped it up quickly and needs double-checking)
• [ ] Use an actual sample dataset
• [ ] Visualize clusters
• [ ] Think through clustering ITP and re-factors into library using deeplearn.js for vector math

There are some interesting possibilities with combining clustering algorithms with word vectors.

[shiffman]

Any ideas for a good sample dataset to try with this? Eventually I'd like to try some interesting demos like @genekogan's amazing TSNe Viewer but will start with something smaller, simpler and more suited for kmeans.

Maybe colors or small corpus of word vectors?

[genekogan]

hi @shiffman! i often dig into the (now relatively old) CalTech-256 which is structured into 256 rather random categories. the tsne demo just is the subset of those categories which are animals. it's neatly organized so it's easy to make themed subsets.

[vndrewlee]

The iris dataset is popular for kmeans demos. There are only four dimensions so it's relatively simple to visualize the clusters.

If the intended audience is new to machine learning, the results can be visualized without additional concepts such as PCA.

also, I believe the example is now here: https://github.com/ml5js/ml5-examples/tree/master/p5js/WIP_clustering/kmeans

[shiffman]

Adding to this thread a good reference for t-SNE by @enjalot

https://distill.pub/2016/misread-tsne/

[shiffman]

Amazing reference! Realtime tSNE Visualizations with TensorFlow.js by @Nicola17

https://twitter.com/nicolapezzotti/status/1004866454578257922?s=11 https://ai.googleblog.com/2018/06/realtime-tsne-visualizations-with.html

[genekogan]

this looks really nice. great find!

[cvalenzuela]

We could work on porting this:

https://github.com/tensorflow/tfjs-tsne

[vndrewlee]

I had originally made a pull request for this, but development fell off and I closed the pull request. I'm leaving some breadcrumbs here for reference.

• pull request
• branch
• observable notebook demo

[jwilber]

I can work on some of these over summer - is there a particular API I should strive for/methods I should expose?

[joeyklee]

Hi @jwilber - Hooray! Thanks so much for your interest. This would be super wonderful.

I'm happy to chat more about the API structure anytime (I know @shiffman and @yining1023 will have great feedback here), but in general, the ml5 process goes something like:

1. load a model & data
2. classify(), segment(), generate() etc... in this case maybe .cluster() ?
3. do something with the results

So a rough rough proposal might be something where you set a bunch of options and the kmeans just spits out a bunch of results:

const options = {
  prop1: 1,
  prop2: 'something'
}

const dataUrl = 'rainbowData.csv'
const kmeans = ml5.cluster('kmeans', dataUrl, options, modelWithDataLoadedCallback)

function modelWithDataLoadedCallback(err, data){
  if(err){
   return err;
 }
  console.log(results)
)}

Maybe a different approach could be something like:

const options = {
  prop1: 1,
  prop2: 'something'
}

const dataUrl = 'rainbowData.csv'

const kmeans = ml5.cluster('kmeans', dataUrl, dataLoadedCallback)

function dataLoadedCallback(err, _data){
     if(err) return err;

    kmeans.classify(data, options, dataCrunchedCallback)
}

function dataCrunchedCallback(err, data){
     if(err) return err;
     // do something with the data
}

I'm not sure yet what the terminology or function naming would be best here quite yet, but in general, we try to use more approachable terms or helpful analogies.

I also wonder if it makes sense if some of these functions become "helper" functions like:

ml5.helpers.kmeans()

turf.js has a kmeans implementation for geo operations. Maybe there's some helpful nuggets in there for us too?

[jwilber]

@joeyklee thanks for the detailed response! I won't be able to get started until roughly the end of the month, so expect an update around then :)

Also - I'm assuming I should implement each alg using tensorflowjs? Is there a particular version? Thanks!

[joeyklee]

@jwilber - Sure thing! Thanks so much for your interest in ml5! This issue has been open awhile and we'd love to open up these kinds of methods to the ml5 community. No pressure on timing! Also nice to enjoy the summer :)

Also - I'm assuming I should implement each alg using tensorflowjs?

• Yes and no. While we've noticed that sometimes using tensorflow can be overkill, the nice thing is that we can have more consistent implementations for things as well as reduce the need for more dependencies. However, sometimes it is better not to use a "screwdriver to hammer in a nail" ;)

Is there a particular version? Thanks!

• Currently we are using tensorflow.js version 1.1.2. Word on the street is that version 2.0 is coming in with some big changes so maybe this is something to keep in the back of our minds. As an aside: We have a dependency to tensorflow/magenta: https://github.com/tensorflow/magenta which is currently using tensorflow version 1.0.2 so we should try to be aware to not diverge too far if possible.

p.s. Your pudding.cool piece on skate music is one of my favorite data viz/editorial pieces. I can still remember every trick on beat for all skate videos between ~2000 - 2010. I love the line Classic rock is often used with a so-called “hesh” style of skateboarding, 😂

[jwilber]

Haha, thanks a lot! Skateboarders of the world unite ✊. (I actually watched Appleyard's part in Sorry recently and oh, man, the waves of nostalgia I felt when the Placebo song came on were insane).

Re: implementation: As a first pass I'll just grab any relevant functions I need from tf.js, and if it only turns out to be a few we can discuss cleaning up the dependencies (i.e. rewriting them as class methods or whatever) in the PR.

Looking forward to contributing :)

[joeyklee]

@jwilber :

• Yes! 🤙 - The part that always gets me is Pappalardo's part in Mosaic. The twangy guitar intro mixed w/ the b-roll images were so nicely cut to Dinosaur Jr!

Re implementation:

• Sounds perfect!
• For me the nicest thing would be to wrap up the relevant tf.js functions in a friendly way and show a few examples of how and when different clustering methods might be handy.
• We'd certainly welcome PRs to the ml5-examples repo in case you had some simple but illuminating ways to showcase in which contexts how kMeans, for example, might be useful. Any ideas are welcome!

Thanks!

[jwilber]

Sorry for long response (had heart surgery and it took longer to recover than I thought) - I'll start working on this over the weekend, so expect a first pass next week!

[joeyklee]

@jwilber - Thanks for the update + no worries. I hope you have a speedy recovery! As many of the ml5 team have been on holidays (myself included) we're also getting caught up with issues and PRs etc.

Thanks + take care!

[joeyklee]

Just making a note that the KMeans class is now part of the development branch via @jwilber ✨

[joeyklee]

Linking to: https://github.com/ml5js/ml5-library/issues/563

[bomanimc]

Thanks for adding this class! Closing for now, since we have a separate issue for DBScan.