search

awslabs / handwritten-text-recognition-for-apache-mxnet

This repository lets you train neural networks models for performing end-to-end full-page handwriting recognition using the Apache MXNet deep learning frameworks on the IAM Dataset.
Apache License 2.0
481 stars 189 forks source link

Is K-fold crossvalidation happening? #26

Closed man0007 closed 4 years ago

man0007 commented 4 years ago

HI,

First, let me appreciate your great work in sharing this handwriting model as open source!

I just understood that you are splitting the dataset into fixed validation set for validation.

Will it be more efficient to implement k-fold cross-vaildation, so that the we can increase the amount of data used for training.

One more question, Will it be okay to use the entire dataset for testing? or should it be necessary that I should only use the unseen training dataset for testing the model?

Thanks, Anand.

jonomon commented 4 years ago

Hi Anand, thank you for your comment.

The train and test split for this was based on the definitions provided http://www.fki.inf.unibe.ch/databases/iam-handwriting-database. It may be beneficial to implement a k-fold cross validation scheme but we felt that it was more important to follow the scheme provided by the dataset.

man0007 commented 4 years ago

HI Jonathan,

Thanks for your quick response!

Yesterday I was trying to run a script inside OCR folder (handwriting_line_recognition.py), when i did I could see that the training_dataset for this model is created as a combination of the data present in the files trainset, val1, val2, which are mentioned inside subjects folder (dataset folder -> Iamdataset -> subject).

Why the val1 and val2 data are merged with training dataset?

Here only two types of loss calculation is happening: train_loss at the time of training, and testing_loss at the time of testing. Why are we not maintaining the validation set separately for validation loss calculation in here?

Is the model weights are updated based on the testing _loss only, since there is no validation_loss calculation in here? If so will it not introduce bias if we use testing_loss for updating weights?

Thanks, Anand