No straightforward way to train on expected data rather than labels, in C++ API

[gh2k]

Looking at the C++ API documentation at: https://github.com/hughperkins/DeepCL/blob/master/doc/NeuralNetAPI.md

I see an example using NetLearner to train on labels, but I don't see an equivalent way of training on expected data (floats)

Can you point to some example code on how to do this? Is it possible to extend NetLearner to provide support for expected data?

[hughperkins]

swap the SoftMax for SquaredLoss, ie: ``` net->addLayer( SquareLossMaker::instance() );

[hughperkins]

Ah, you mean, the NetLearner doesnt accept floats? Hmmm, let me check...

[gh2k]

Yeah, I got as far as the SquareLossMaker as described in the API docs, and I've got all my floats packed into what's described as the data format on the same page. There's just no way to pass a float array to NetLearner.

[hughperkins]

Hmmm, seems you are right: all the batch helper classes only accept labels. But ... I remember I created a class to generalize labels vs expected data, which is ... file BatchData.hhttps://github.com/hughperkins/DeepCL/blob/master/src/batch/BatchData.h which has a base classOutputData, and two child classes:ExpectedData(ie what you want), andLabelledData. So, anywhere that accepts anOutputData in theory implicitly acceptedExpectedData . Then.... searching through the files inbatchdirectory, there are a few classes that acceptOutputData *`:

• some classes in NetAction2 https://github.com/hughperkins/DeepCL/blob/master/src/batch/NetAction2.h , and
• Batcher2 class https://github.com/hughperkins/DeepCL/blob/master/src/batch/Batcher2.h

Looking at Batcher2, it takes in:

• a NetLearner object, which decides are we doing forward prop only (NetForwardAction2), or training, forward/backward prop (NetLearnAction2) (both are in https://github.com/hughperkins/DeepCL/blob/master/src/batch/NetAction2.h )
• an OutputData *, to which we pass our ExpectedData * object
• an InputData * object, for hte input data
• the Net * that we want to train

... and then you just need to call tick() on this a bunch of times, I think, check getEpochDone() after each tick, and call reset() after each epoch

There is also LearnBatcher2 child class, which provides the NetLearnAction2 for you https://github.com/hughperkins/DeepCL/blob/master/src/batch/Batcher2.h#L67

So, that will handle running batches on all your data. If your data fits in memory, I think that's all you need.

Unfortunately, if you need to handle loading a bit of data at a time, since the data wont fit in memory, it looks like you'll need to handle that yourself, calling out to eg LearnBatcher2 for each set of batches. There are wrappers such as https://github.com/hughperkins/DeepCL/blob/master/src/batch/NetLearnerOnDemandv2.h , but it looks like they dont use the LearnBatcher2 etc, and all assume labelled data, for now.

[gh2k]

My data should fit in memory, so I'll check out LearnBatcher2...

If I can make it work I'll try and update the docs.

[hughperkins]

Ok, cool. Let me know how that goes please