💡[Feature]: Code for optimizer

[ojhankit]

Is there an existing issue for this?

• [X] I have searched the existing issues

Feature Description

The work I propose involves:

Implementing key optimizers such as Stochastic Gradient Descent (SGD), Adam, RMSprop, and others from scratch. This would allow for comparison between different optimization algorithms in your models

Documenting each optimizer and explaining their strengths, weaknesses, and use cases, making it easier for users to choose the right one.

Use Case

1.model performance tuning 2.Better understanding on choosing optimizer

Benefits

No response

Add ScreenShots

No response

Priority

High

Record

• [X] I have read the Contributing Guidelines
• [X] I'm a GSSOC'24 contributor
• [X] I want to work on this issue

[github-actions[bot]]

Thank you for creating this issue! 🎉 We'll look into it as soon as possible. In the meantime, please make sure to provide all the necessary details and context. If you have any questions reach out to LinkedIn. Your contributions are highly appreciated! 😊

Note: I Maintain the repo issue twice a day, or ideally 1 day, If your issue goes stale for more than one day you can tag and comment on this same issue.

You can also check our CONTRIBUTING.md for guidelines on contributing to this project.
We are here to help you on this journey of opensource, any help feel free to tag me or book an appointment.

[ojhankit]

@sanjay-kv please do check my idea and if you find this satisfactory please assign this to me.