search

Fraunhofer-IMS / AIfES_for_Arduino

This is the Arduino® compatible port of the AIfES machine learning framework, developed and maintained by Fraunhofer Institute for Microelectronic Circuits and Systems.
GNU Affero General Public License v3.0
216 stars 45 forks source link

Linear regression training problem #9

Open s1498n089 opened 2 years ago

s1498n089 commented 2 years ago

Hello, I would like to ask some questions, forgive my English is not good. When I was doing the linear regression training, the value of the Loss function appeared to be fixed at a certain value. For my case, it will always be 14.25. Of course, the trained model did not succeed, and the the predicted value of the model before training is also always 0. But after several failures, I accidentally touched the analog pin during setup(), and then the training was successful. The following is the log of my training model, one succeeded (random seed: 641), the other failed (random seed: 17) Training fail: training_fail.txt

Training success: training_success.txt

I think it should be the random seed that caused the initial model weight setting problem. Is my idea correct?

PierreGembaczka commented 2 years ago

Hi @s1498n089,

I think that your learning rate is too high (Try 0.01 or 0.001 and increase slowly). What value are you using right now?

If the learning rate is too high, you may not find the global minimum. Then you have to be lucky with the initialized weights.

In the examples (XOR) the learning rate is set very high, because the models are quite easy to train.

Best, Pierre

s1498n089 commented 2 years ago

Hi @AIfES-Pierre, Thanks for your answer. My learning rate is obtained from the XOR example. I did not change it, but kept it at 0.1, and when I followed your suggestion, I adjusted it to 0.01 or 0.001, that fixed value problem is solved, thank you very much.