Supervised Classification Example

[skn123]

Is there any example for supervised classification of feature vectors? Can you please share some pointers?

[hughperkins]

Yes sure, so:

• if you're using python you can have a look at https://github.com/hughperkins/DeepCL/blob/master/python/test_deepcl.py
• if you're using C/C++, you can have a look at https://github.com/hughperkins/DeepCL/blob/master/src/main/train.cpp#L357-L423 A bit complicated though, I confess. So ... hmmm.... you know, originally this was the simple example :-) But it kind of evolved with time... maybe I should make a dedicated simple example
• from commandline, you can see for example https://github.com/hughperkins/DeepCL/blob/master/doc/Commandline.md#training

What language are you hoping to use to run the training? ie, python? C/C++? commandline? (something else?)

[skn123]

Hugh Thanks for the response. I work with C++. So I started off with looking at train.cpp / test.cpp from command line. So an ideal example for the "uninitiated" would be as follows: a.) A simple example of populating a set of Multi-class training feature vectors in DeepCL's internal format b.) Generating a training model c.) Serializing the model to a std::string and saving it to disk d.) Loading the model from disk as a std::string and deserializing it e.) Classify or regress a set of test features using this model.

Do you have any example to this effect?

[hughperkins]

Somehow this fell through my radar. I should address this...

[skn123]

Hugh, any updates on this?

[hughperkins]

Well, lets go one step at a time, otherwise I just stare at a huge long list, and go ..' errr .... some time, maybe' :-P

So, 'a.) A simple example of populating a set of Multi-class training feature vectors in DeepCL's internal format'. A few places you can get this from:

• https://github.com/hughperkins/DeepCL/blob/master/src/main/train.cpp#L177-L194
• this calls GenericLoaderv2, which handles a variety of formats
• actually, this is just a wrapper around various per-format loaders. You can see the wrapping bit here https://github.com/hughperkins/DeepCL/blob/master/src/loaders/GenericLoaderv2.cpp#L33-L39
• GenericLoaderV1Wrapper is just a loader around the earlier GenericLoader https://github.com/hughperkins/DeepCL/blob/master/src/loaders/GenericLoaderv1Wrapper.cpp#L41
• GenericLoader (ie the v1 loader) handles a bunch of formats https://github.com/hughperkins/DeepCL/blob/master/src/loaders/GenericLoader.cpp#L37-L48
• ManifestLoader is for reading huge directories of images, ie for imagenet, with a single manifest file describing the whole tree of images https://github.com/hughperkins/DeepCL/blob/master/src/loaders/ManifestLoaderv1.cpp

So, pick one of these loaders, and look how it works. Alternatively, basically there are two sets of data:

• input data features. These are in a 1-dimensional vector, laid out as [n][plane][height][width] (but in 1 dimension, but the strides are in this order)
• labels. These are also a 1-dimensional vector, of labels (0-based, I think)

[hughperkins]

alternatively, really, skip all the file links, and just jump to the last couple of sentences. You need two 1-dimensional arrays, as stated above:

• input data features. These are in a 1-dimensional array of floats, laid out as [n][plane][height][width] (but in 1 dimension, but the strides are in this order), ie a float *
• labels. These are also a 1-dimensional array, of labels, of type int *

[hughperkins]

(really, everything you need is in https://github.com/hughperkins/DeepCL/blob/master/src/main/train.cpp , its just a question of picking out the bits you need really. for example, this handles a bunch of different trainers https://github.com/hughperkins/DeepCL/blob/master/src/main/train.cpp#L287-L319 but you only need to pick one. For example, if you want to use SGD, then use these lines https://github.com/hughperkins/DeepCL/blob/master/src/main/train.cpp#L289-L293 )

[hughperkins]

as far as 'serializing to a string', this would be new functionality. you can save to disk, but it would be in binary format. It's handled by this line https://github.com/hughperkins/DeepCL/blob/master/src/main/train.cpp#L417-L418 , so you can follow the breadcrumbs, and have a look at how it works. If you want to convert to a string, you could for example copy and paste the WeightsPersister, to a new class, and modify it to your requirements.

[hughperkins]

As far as 'classify', you can look at how the 'predict' works, ie https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp Again, you'll need to kind of pick out the bits you need, and throw the rest away. For example everything after line 287 https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp#L287 is just about reading commandline options, printing usage etc, so you can probalby just ignore that. Otherwise, we can see we:

• call genericloader, as before, to load the data https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp#L109
• Create the network/model using NetdefToNet https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp#L152
• Loop over data https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp#L215
• forwrad through each layer of the network https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp#L220-L223
• cast last layer to SoftMaxLayer, and get the labels out https://github.com/hughperkins/DeepCL/blob/master/src/main/predict.cpp#L244-L249