search

trevorstephens / gplearn

Genetic Programming in Python, with a scikit-learn inspired API
http://gplearn.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
1.6k stars 282 forks source link

force rational functions rather than polynomials #38

Closed guyko81 closed 6 years ago

guyko81 commented 7 years ago

Hi @trevorstephens ,

I still love your work and found a good paper: Neural networks and rational functions It describes that rational functions can approximate arbitrary functions better than polynomials. That made me think that the gplearn should generate 2 separate polynomials in parallel and use the ratio of them as the function approximate. My argument is the following (without exactly knowing the proof of theorems):

  1. it is proven that a neural network with non-linear activations can approximate any function
  2. it is proven in this paper that a rational function (functions represented as the ratio of two polynomials) can approximate any RELU neural network (which is nonlinear) ==>
  3. so the rational functions can approximate any functions

I'm not sure that the performance or the size will be optimal, but I think it worth a try. What do you think?

guyko

guyko81 commented 7 years ago

I think the only modification (or option) would be to hold the first branch on the '/' value (DIVIDE)

trevorstephens commented 6 years ago

Hi @guyko81 , thanks for the suggestion... I think at 3 citations it will need more support for me to integrate this into gplearn though.