search

baaivision / Emu3

Next-Token Prediction is All You Need
Apache License 2.0
874 stars 26 forks source link

cannot inference on A100 40GB #20

Open quang-ngh opened 1 day ago

quang-ngh commented 1 day ago

Hi,

When i try to run text-to-image generation with single A100 40GB, it is out of memory at the decoding phase (with VQModel). Is that unusual ?

Thanks

ryanzhangfan commented 1 day ago

We have verified our inference code on A100-40G and it do not raise OOM issues. You can try using torch.cuda.empty_cache() before decoding or make sure using torch.no_grad() during inference.

quang-ngh commented 1 day ago

Thanks

On Tue, 8 Oct 2024, 09:07 Fan Zhang, @.***> wrote:

We have verified our inference code on A100-40G and it do not raise OOM issues. You can try using torch.cuda.empty_cache() before decoding or make sure using torch.no_grad() during inference.

— Reply to this email directly, view it on GitHub https://github.com/baaivision/Emu3/issues/20#issuecomment-2398502984, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASTXQ2WOGIHRXXUVFXH4SVTZ2M46NAVCNFSM6AAAAABPPO2Z46VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJYGUYDEOJYGQ . You are receiving this because you authored the thread.Message ID: @.***>