search

Lightning-AI / pytorch-lightning

Pretrain, finetune ANY AI model of ANY size on multiple GPUs, TPUs with zero code changes.
https://lightning.ai
Apache License 2.0
28.54k stars 3.39k forks source link

Tuner.scale_batch_size(max_val=1024) #20364

Open edmcman opened 1 month ago

edmcman commented 1 month ago

Description & Motivation

I'm always frustrated when scale_batch_size runs on a CPU machine, and ends up testing horribly large batch sizes like 2^25.

Pitch

I'd like to be able to pass an upper bound batch size as an argument. Batch sizes larger than the argument will never be tested.

Alternatives

No response

Additional context

No response

cc @borda

lantiga commented 1 week ago

That's a great idea, would you like to contribute a patch?

edmcman commented 1 week ago

Sure, I'll give it a shot.

On Mon, Nov 18, 2024 at 7:38 PM Luca Antiga - @. @.> wrote:

That's a great idea, would you like to contribute a patch?

— Reply to this email directly, view it on GitHub https://github.com/Lightning-AI/pytorch-lightning/issues/20364#issuecomment-2484465914, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHYKZKOLDIKLRSB7ZYS3WL2BKB7DAVCNFSM6AAAAABQRVK6OGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBUGQ3DKOJRGQ . You are receiving this because you authored the thread.Message ID: @.***>