ml.dtw.maxhelp example not working

[jarmitage]

We've been porting this help file to Pd: https://github.com/cmuartfab/ml-lib/blob/develop/documentation/help/max/ml.dtw.maxhelp

ml.dtw-help.pd

When following the steps in the comments, Max & Pd both post: error: ml.dtw: unable to map input

We have managed to get it working by randomly clicking on stuff, so we know it can work but something about this example workflow is wrong. Any ideas? Do some of the attributes need to be set before training?

[jarmitage]

Seems to be duplicate/related to #80 #133

[jamiebullock]

Hi. Yup, this is a duplicate of #80 and #133.

The error in the help file is that "record" needs to be toggled on / off at each gesture exemplar boundary. So the correct steps should be:

0. Clear
1. Turn record ON
2. Sample ONE
3. Turn record OFF
4. Turn record ON
5. Sample TWO
6. Turn record OFF
7. Train
8. Turn record ON
9. Test ONE
10. Turn record OFF
11. Turn record ON
12. Test TWO

i.e. "record 0" "record 1" is used as a delimiter for each gesture

Does that make sense?

If you are porting help files it would be awesome if you could update the help and submit a pull request.

[jarmitage]

Thanks for confirming. I have a fork with a pd-help branch where we’re committing so if anything good comes of it I’ll PR.

I’m a bit confused about steps 8-12 there. Does ml.dtw not work in a ‘live’ context, and do you have to toggle record for every gesture?

[jamiebullock]

Does ml.dtw not work in a ‘live’ context, and do you have to toggle record for every gesture?

It will attempt to work continuously if you just leave record on whilst mapping, however you will obviously get more misclassifications due ambiguity between class boundaries, local minima etc

It is better to segment your data somehow so you are only asking DTW "what class is this gesture?" rather than "find gesture X within this stream", which is a harder problem. You could potentially automate the segmentation with another algorithm such as SVM, or even just use a simple heuristic.

[jarmitage]

Thanks that's really good to know. We're finding that this sort of knowledge about "how to use which algorithm when" is critical, is this covered in the ml.lib/GRT literature, or somewhere else? Or should we be thinking about a means to collate that?

And would I be right then in thinking that DTW is only suited for post-hoc classification? If so are there other algorithms in ml.lib or elsewhere that we should look at for streaming data?

[jamiebullock]

It's not post-hoc in the sense that you record input then map it and get the result. Whilst record is on DTW will attempt to continuously classify the input, so you get an output class for every observation. It's just that as you add more observations the classification may change, and become more accurate.

For example, say you have three classes of input:

CLASS 1: 1 2 1 2 3 1 2 3 4 CLASS 2: 1 2 1 2 3 2 1 0 0 CLASS 3: 1 2 1 2 3

You will get an output class for each value in the input, but up to the 5th observation, they are the same, so there is no way to differentiate the data until the 6th observation is added.

However, if you add an additional class:

CLASS 4: 5 4 3 2 1

The classifier will probably give the "correct" classification of 4 after only one observation "5"

So it is not really post-hoc as such, it is just that in order to disambiguate classes 1–4, some segmentation is advised.

In terms of "how to use", there is very little direct documentation. There are some academic papers, but I think some clear user facing documentation would be a strong contribution.