Classify using a neural network

[johannadevos]

I'm going to start doing this tutorial: https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/

If I get that working, hopefully I can use the same code for our binary classification task.

[johannadevos]

This is useful too: https://www.tensorflow.org/get_started/premade_estimators

[johannadevos]

I got the neural network working in a rudimentary version, where it trains on 100 reviews, using two layers with 10 hidden nodes each (otherwise it wasn't feasible on my laptop). The code is extremely unpolished still (in the branch johanna_NN, part1.py), but it runs through.

It is not obvious to me how I should integrate this code in bagofwords.py. In the optimization_run function we use things such as classifier.fit and classifier.predict, but the fit and predict methods belong to the RandomForestClassifier class, and not to the DNNClassifier class, which for example works as:

classifier2.train(
    input_fn=lambda:train_input_fn(train_df, train_data['sentiment'], 100), # batch size 100
    steps=1000)

eval_result = classifier2.evaluate(
    input_fn=lambda:eval_input_fn(test_df, test_data['sentiment'], 100))

What would be the best way to go forward and integrate this code?

[johannadevos]

Nastaran's tips:

• The number of samples needs to be at least 3 times higher than the number of parameters.
• The number of parameters currently is 100 (input layer ) x 10 x 10 x 2 (output layer) + 10 (bias hidden layer 1) + 10 (hidden layer 2) + 2 (output layer).
• So we have about 20.000 parameters but only 25.000 samples.
• This tutorial is useful: https://machinelearningmastery.com/predict-sentiment-movie-reviews-using-deep-learning/
• They use a convolutional neural network, which has fewer parameters. We should use this architecture, which tries to learn the features, instead of us extracting manual features. This means we give the raw data (after preprocessing) to the neural network.
• The output of this CNN can be used as features for a classifier, for example a random forest, or continue using these features in a neural network that ends in binary classification.
• This book is useful: https://github.com/janishar/mit-deep-learning-book-pdf
• Also, see the Keras documentation: https://keras.io/layers/convolutional/