TensorFlowLinearRegressor unable to recover weights?

[cwharland]

I'm hoping that I just overlooked something but I can't seem recover the correct weights for simple linear test cases using TensorFlowLinearRegressor. Here's a gist of the full test file (with pytest) and matching test case using scikit-learn LinearRegression. As the comment in the gist says, the scikit module recovers the weights faithfully while skflow returns odd weights.

Package details: Python 3.5 tensorflow 0.6.0 skflow 0.0.1

I coded up a quick linear regression module myself using tensorflow as backend (using the SGD optimizer) and it was able to recover the weights using the same iterations, learning rate, etc... as default TensorFlowLinearRegressor so I'm curious what I'm doing wrong with skflow (or the test case) or if there's an issue somewhere.

Thanks for the comprehensive work on the library. Looking forward to using it a bunch.

[ilblackdragon]

Hi,

I just took a quick look. If you set batch_size to 1 and increase number of steps, it actually passes:

regressor = skflow.TensorFlowLinearRegressor(steps=2000, learning_rate=0.1,
    batch_size=1)

I'll look into more, why with the bigger batch size it doesn't converge properly.

Thanks for reporting.

[cwharland]

Ahh good catch. In my tensorflow backed quick mash up I had implicitly set batch size to 1 by making the fit() method run the session for each data point. So yeah there's something going on with batch gradient descent. If I find anything else that helps even out the discrepancy I'll post back.

[ilblackdragon]

So, there was the wrong shape for labels in the loss computation and it was broadcasting, creating a square matrix from diff. Fixed with commit above. Please let me know if it look good now!