Use the non-protected LRScheduler import

carmocca commented 2 years ago

🚀 Feature

Motivation

Avoid protected imports

Pitch

https://github.com/pytorch/pytorch/issues/61232 has been merged to PyTorch which renames _LRScheduler to LRScheduler. They have kept the old class for compatibility, but we should still use the new one.

The task is to add logic like this

LRScheduler = (
    torch.optim.lr_scheduler.LRScheduler
    if _TORCH_GREATER_EQUAL_1_14 else 
    torch.optim.lr_scheduler._LRScheduler
)

to https://github.com/Lightning-AI/lightning/blob/d5003b1c07fda783f651a732c86ad48656be42c1/src/lightning_lite/utilities/types.py#L66 and places that use it

Alternatives

Keep using the protected import

If you enjoy Lightning, check out our other projects! ⚡

Metrics: Machine learning metrics for distributed, scalable PyTorch applications.
Lite: enables pure PyTorch users to scale their existing code on any kind of device while retaining full control over their own loops and optimization logic.
Flash: The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, fine-tuning, and solving problems with deep learning.
Bolts: Pretrained SOTA Deep Learning models, callbacks, and more for research and production with PyTorch Lightning and PyTorch.
Lightning Transformers: Flexible interface for high-performance research using SOTA Transformers leveraging PyTorch Lightning, Transformers, and Hydra.

cc @borda

qmaruf commented 2 years ago

I am interested to work on this feature. @carmocca What are the things I need to follow?

carmocca commented 2 years ago

I described the work in the top post. We want to use LRScheduler throughout the codebase by importing a small compatibility variable defined in lightning/src/lightning_lite/utilities/types.py

qmaruf commented 2 years ago

Hi @shenoynikhil, i am working on this one.

qmaruf commented 2 years ago

All the changes are here https://github.com/Lightning-AI/lightning/compare/master...qmaruf:lightning:feature-15587

However, some tests are failing because of this issue RuntimeError: torch.distributed is not available. Cannot initialize distributed process group. I am using mac m1 pro.

cc @carmocca

carmocca commented 2 years ago

Thanks! You can go ahead and open a PR

ahmadmustafaanis commented 2 years ago

@carmocca can you also point me to a good first issue on which I can work? I see on most of the issues there are people already working.

Lightning-AI / pytorch-lightning