[Feature] Integration of Existing Knowledge

[hsianktin]

First, thanks for the great package! I'm trying to adopt known formula to simplify/facilitate the procedure of symbolic regression. For example, when $\varepsilon \ll 1$, we obtained a first order approximation of some function $f(x, y)$, where $x,y,\varepsilon$ are all parameters. The output is denoted as $output=f(x,y)$ and the test data are called $validation$, there is a loss function $\ell(output,validation)$. Later, I determined that we may fit a function $\xi(x,y,\varepsilon)$ to extrapolate the first order approximation by parametrizing $x$ in the arguments of $f(x,y,\varepsilon)$: $\tilde f(x,y,\varepsilon) = f(\xi(x,y,\varepsilon), y).$

If $y$ is fixed, such knowledge of $f$ can be distilled into a new loss function $\tilde \ell(output,validation) = \ell (f(output), validation)$. Then, fitting output $\xi$ is equivalent to find $f(\xi(x,y,\varepsilon), y).$ $\xi$ is in general easier to fit because it is close to $x$.

However, if $y$ changes, this trick does not seem to work. One probably easy-to-implement way I can figure out to make all this work, is to explicitly allow the loss function $\ell$ to "peek" into original input $(x,y,\varepsilon)$. For example, in the sample loss function below, it accepts original input $y$ as an optional argument.

function new_loss(output,validation, y=y0)
    return loss(f(output,y), validation)
end

I guess this could probably be done by modifying eval_loss function to pass dataset.X to the loss function if requested. But I don't quite know the underlying details.

[MilesCranmer]

Hey, sorry for the late reply - I missed this issue somehow.

I think this pull request: https://github.com/MilesCranmer/SymbolicRegression.jl/pull/143 will fix this? You can try it out by checking out that pull request and setting it up manually. Also see my comment here: https://github.com/MilesCranmer/SymbolicRegression.jl/pull/143#issuecomment-1279205103 which might provide a general way of incorporating prior knowledge.

Cheers, Miles

[hsianktin]

Yes, exactly.

Thanks a lot! Xiangting