Tuning Pytorch custom LSTM with Optuna

[amine759]

Motivation

I had to adjust a custom LSTM for my sequential dataset to fit my needs. I thought of creating an example of how to implement this adjustment, that it could be helpful and save time for others who might need to make similar adjustments to their own custom LSTMs.

Description of the changes

The added file shows how to handle a custom forward pass for training LSTM with the standard model definition in Optuna, with the define_model and objective functions, provided in simple pytorch examples .

[nzw0301]

Thank you for your PR. This repo has already provided several PyTorch examples, so I'm not sure that this LSTM example is necessary for this repo. Anyway, I'd like to hear the opinion from the core-dev team.

[github-actions[bot]]

This pull request has not seen any recent activity.

[toshihikoyanase]

@amine759 Thank you for your PR! I have a question about the LSTM model and training recipe. Do they follow official PyTorch examples? Or are they based on your own code? Just a curiosity, but I usually follow the official examples of deep learning libraries to create Optuna examples.

@nzw0301 Thank you for your comment. I'll ask the other members as well.

[amine759]

@toshihikoyanase Thank you for your reply, this PR does not rely on any PyTorch built-in datasets (like you usually do😄) . It assumes the use of a preprocessed sequential data, such as a time series. It is designed to optimize models with custom forward passes, if a user choose to do so.

[toshihikoyanase]

@amine759 Thank you for your reply. I understand that your example, including both the dataset and model, does not rely on the official PyTorch example. In this case, its usefulness for readers might be limited. This is partly because users cannot refer to the detailed explanation of LSTM learning provided in the official example, and also because this repository does not offer such detailed documentation for each example.

Given these circumstances, I believe it might be more beneficial for you to write a blog post about your example, where you can share your knowledge of LSTM in detail. What do you think of it?

[amine759]

@toshihikoyanase I comprehend your point that this particular example is very specific and deviates from the typical examples found in this repo. Creating a blog post on this subject seems like a beneficial suggestion. In any case I appreciate your reaction!

[toshihikoyanase]

Thank you for your understanding regarding the example and the repository's policy. Once you've written your blog post, please consider sharing it on social media so that it can reach a wider audience of engineers and researchers who use LSTM. I'm looking forward to reading it and learning from your insights.

[github-actions[bot]]

This pull request has not seen any recent activity.