huggingface / optimum-neuron

Easy, fast and very cheap training and inference on AWS Trainium and Inferentia chips.
Apache License 2.0
176 stars 51 forks source link

Disable weights / neff separation of SDXL's UNET for neuron sdk 2.18 #554

Closed JingyaHuang closed 2 months ago

JingyaHuang commented 2 months ago

What does this PR do?

Issue reported here: https://github.com/aws-neuron/aws-neuron-sdk/issues/859

Force inline_weights_to_neff=True for sdxl's unet for now to be able to bump optimum-neuron to neuron SDK 2.18, to remove after it's patched by the Annapurna team.

Before submitting

HuggingFaceDocBuilderDev commented 2 months ago

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.