implement NN structure implemented during undergrad thesis

[atkinssamuel]

See https://github.com/atkinssamuel/cryptonet-research/blob/master/BaselineTiming/Models/Encrypted/encrypted_test.py

The summarize the nature of this repo:

• The problem being addressed was a regression problem
• There was a linear layer (15 -> 30) with a squared activation function followed by another linear layer (30 -> 1) with no activation function
• The Adam Optimizer was utilized
• The sum of least squared errors was minimized

[atkinssamuel]

I will be implementing the following:

• Classification problem
• LR model with approximate sigmoid activation function
• Least squared error loss function
• NAG optimizer
• Xavier initialization

Edit: Implemented the above. Unfortunately, the loss does not converge even when a rescaling layer is used. Will try some other ideas tomorrow.

[atkinssamuel]

So far I have not been able to get the squared activation function to work under any circumstances. A linear regression, however, can be done in the encrypted domain so that may be a good place to start.