Some functions in the Regression module return wrong results

[igarnier]

Some functions in the Regression module return wrong results for some (most?) inputs. An example is provided below:

module R = Owl.Regression.D

let input =
  Owl.Arr.uniform ~a:0.0 ~b:10.0 [| 30; 1 |]

let c = 20.0

let b = 40.0

let output =
  Owl.Arr.init [| 30; 1 |] (fun i ->
      (Owl.Arr.get input [| i; 0 |]) *. c +. b
    )

let w, bias =
  let res = R.ols ~i:true input output in
  res.(0) , res.(1)

(* The following print values w ~= 22.19 and bias ~= 25.58 *)
let _ =
  Owl.Arr.print w

let _ =
  Owl.Arr.print bias

[mreppen]

This is due to the GD not converging. Bumping the number of iterations gives 20 and 40.

For methods that do not guarantee convergence, there should be some indicator of success. Either some special return type, or just something wrapped as Result.t

@ryanrhymes Any thoughts?

[igarnier]

Hi, I'm looking again at this issue. What is the currently implemented method for checking termination of gradient descent? Are we looking at the delta of the loss function over the last iteration? Shouldn't it be possible to have a criterion that always works for problems such as least squares?

[mreppen]

I wasn't involved in writing it and haven't looked since my last post, but this seems to be the implementation: https://github.com/owlbarn/owl/blob/85c6e06deec7963b7222553ec28aedef8bb1b94f/src/owl/optimise/owl_regression_generic.ml#L38-L49

It does have a criterion. What happens in your example is that the maximum number of iterations is reached.

Arguably, without penalty, it is better to move to another algorithm instead of tailoring the stopping condition.