AtheMathmo / rusty-machine

Machine Learning library for Rust
https://crates.io/crates/rusty-machine/
MIT License
1.25k stars 152 forks source link

Improvements to linear regression #5

Closed AtheMathmo closed 8 years ago

AtheMathmo commented 8 years ago

The linear regression model is currently incomplete and untidy. The following changes are required:

The linear regression model will see reworking in the future as we implement regularization (ridge regression) and model optimization (lasso, forward/back selection, etc.). These things are currently unmapped but the above changes will be needed regardless.

Please comment if you see other things which should be changed. LinRegressor was the first machine learning algorithm implemented in this library and so undoubtedly could use some work!

AtheMathmo commented 8 years ago

I handled the first change on the master branch as the others will take more time and thought but I wanted this one in soon.

raulsi commented 8 years ago

@AtheMathmo about second issue. new rows of ones in train function right. and an additional parameter is the default value will be 1 for now?

AtheMathmo commented 8 years ago

I mistyped, it should be a new column of ones. So if we have an input matrix X, we will produce the matrix [1 X] - where 1 represents a column of ones (with the same number of rows as X).

We'll need to do this change in both the train and predict functions. It will be sufficient to simply update X with something like:

let ones = Matrix::<f64>::ones(inputs.rows(), 1);
let full_inputs = ones.hcat(inputs);

And then use full_inputs in place of inputs for the rest of the scope.

I've added another improvement to the issue above as well (another easy one).

AtheMathmo commented 8 years ago

Closing this issue for now. I want to change how the optimization is handled in the future but for now this will lay the foundation.