search

hunar4321 / reweight-gpt

Reweight GPT - a simple neural network using transformer architecture for next character prediction
https://www.brainxyz.com/
MIT License
47 stars 7 forks source link

neural_network_tutorials.ipynb #1

Open Myrkiriad-coder opened 1 year ago

Myrkiriad-coder commented 1 year ago

Hello,

Big fan of your videos, I decided to look into your code this time !

In neural_network_tutorials.ipynb, 4/ Deep Neural Networks (Numpy).

If I changeys = xs**2, for anything else (ys = xs**3, ys = xs**4, ys = np.sin(xs), ys = np.abs(xs)etc....), there is no fit. Any reason for that ?

hunar4321 commented 1 year ago

Yes, that is expected Every time you change the order of your equation, you must also change the parameters too. Reduce the learning rate if your error plot is noisy. You can also tweak the number of the nodes, iterations, ...etc. Just like what I did here: https://youtu.be/l-CjXFmcVzY?t=872