codeplea / genann

simple neural network library in ANSI C
https://codeplea.com/genann
zlib License
2.01k stars 237 forks source link

Non-linear regression #48

Open ScratchyCode opened 3 years ago

ScratchyCode commented 3 years ago

I'm not skilled with machine learning and i'm trying to study its practical applications. I'm trying to use an ANN for non-linear regression of function sin(x) with analitical points and i have found some problems.

Creating an ANN with 10 hidden layers and 4 neurons, running a for-cycle 1 million times over 100 points maked with x values evenly distributed between 0 and 2pi i obtain this:

alt text

with this specs once created the ANN: ann->activation_output = genann_act_linear; and the output doesn't change trying to make different ANN.

Someone can help me to understand where i'm wrong (also teoretically)? I've seen that sigmoid activation function is not the best choice for this purpose like relu, but it's just this?

ScratchyCode commented 3 years ago

Following the examples of repository my implementation is (most relevant piece of code):

genann *ann = genann_init(INPUT,LAYER,NEURONS,OUTPUT);
ann->activation_output = genann_act_linear;

// training
for(i=0; i<EPOCH; i++){
    for(j=0; j<lenfile; j++){
        genann_train(ann,input+j,output+j,RATE);
    }
}

// first NUMTEST values of dataset
printf("Test on dataset values:\n");
for(i=0; i<NUMTEST; i++){
    printf("f(%lf) = %lf \tresidue = %lf\n",input[i],*genann_run(ann,input + i),output[i] - *genann_run(ann,input + i));
}