Missing predict function

[bmigette]

Hi,

Unless I am mistaking, I don't see any predict function, or any other way to actually use our trained models.

We can only train and evaluate. Am I missing something ?

[bmigette]

Actually, if I understood your code right, it looks like the evaluate function is the one used for prediction. Your project looks to be inspired by Keras, but some parts are confusing: Model.Train is used instead of Model.fit as keras Evaluate is used to predict data, while in Keras, evaluate is to evaluate robustness of your model against a test set. To make prediction, we should have a Predict function.

[deepakkumar1984]

Yes you are correct. Evaluate can be used for evaluating the model using test set as well as for predicting. Internally while training the model with passing the validation set, this method is used on every epoch.

[deepakkumar1984]

And thank you for your kind donation :)

[bmigette]

Hi Deepak, From the author of Keras (Quote from book Deep Learning with Python), validation and test sets are slightly different. Do you plan to have equivalent of keras evaluate function ?

Training, validation, and test sets Evaluating a model always boils down to splitting the available data into three sets: training, validation, and test. You train on the training data and evaluate your model on the validation data. Once your model is ready for prime time, you test it one final time on the test data. You may ask, why not have two sets: a training set and a test set? You’d train on the training data and evaluate on the test data. Much simpler! The reason is that developing a model always involves tuning its configuration: for example, choosing the number of layers or the size of the layers (called the hyperparameters of the model, to distinguish them from the parameters, which are the network’s weights). You do this tuning by using as a feedback signal the performance of the model on the validation data. In essence, this tuning is a form of learning: a search for a good configuration in some parameter space. As a result, tuning the configuration of the model based on its performance on the validation set can quickly result in overfitting to the validation set, even though your model is never directly trained on it. Central to this phenomenon is the notion of information leaks. Every time you tune a hyperparameter of your model based on the model’s performance on the validation set, some information about the validation data leaks into the model. If you do this only once, for one parameter, then very few bits of information will leak, and your validation set will remain reliable to evaluate the model. But if you repeat this many times—running one experiment, evaluating on the validation set, and modifying your model as a result—then you’ll leak an increasingly significant amount of information about the validation set into the model. At the end of the day, you’ll end up with a model that performs artificially well on the validation data, because that’s what you optimized it for. You care about performance on completely new data, not the validation data, so you need to use a completely different, never-before-seen dataset to evaluate the model: the test dataset. Your model shouldn’t have had access to any information about the test set, even indirectly.

[deepakkumar1984]

Can try to implement that mate.

On Fri, Jan 12, 2018 at 7:32 PM, bmigette notifications@github.com wrote:

Hi Deepak, From the author of Keras (Quote from book Deep Learning with Python), validation and test sets are slightly different. Do you plan to have equivalent of keras evaluate function ?

Training, validation, and test sets Evaluating a model always boils down to splitting the available data into three sets: training, validation, and test. You train on the training data and evaluate your model on the validation data. Once your model is ready for prime time, you test it one final time on the test data. You may ask, why not have two sets: a training set and a test set? You’d train on the training data and evaluate on the test data. Much simpler! The reason is that developing a model always involves tuning its configuration: for example, choosing the number of layers or the size of the layers (called the hyperparameters of the model, to distinguish them from the parameters, which are the network’s weights). You do this tuning by using as a feedback signal the performance of the model on the validation data. In essence, this tuning is a form of learning: a search for a good configuration in some parameter space. As a result, tuning the configuration of the model based on its performance on the validation set can quickly result in overfitting to the validation set, even though your model is never directly trained on it. Central to this phenomenon is the notion of information leaks. Every time you tune a hyperparameter of your model based on the model’s performance on the validation set, some information about the validation data leaks into the model. If you do this only once, for one parameter, then very few bits of information will leak, and your validation set will remain reliable to evaluate the model. But if you repeat this many times—running one experiment, evaluating on the validation set, and modifying your model as a result—then you’ll leak an increasingly significant amount of information about the validation set into the model. At the end of the day, you’ll end up with a model that performs artificially well on the validation data, because that’s what you optimized it for. You care about performance on completely new data, not the validation data, so you need to use a completely different, never-before-seen dataset to evaluate the model: the test dataset. Your model shouldn’t have had access to any information about the test set, even indirectly.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/deepakkumar1984/SiaNet/issues/27#issuecomment-357181444, or mute the thread https://github.com/notifications/unsubscribe-auth/AGCQKT3CQ-JFHkqhHw_JGWNdIyteVwZwks5tJx-ugaJpZM4Ra7Pg .

-- Regards, Deepak

[bmigette]

Cool thanks !

[deepakkumar1984]

Closing it off as there is no activity in last 180 days