Disty0
commented
9 months ago Did some more experimenting.
Prior model woks fine with IPEX 2.0 but Decoder model fails. Both models fails with IPEX 2.1.
IPEX 2.0: (Prior Output Preview / Final Decoder Output)
IPEX 2.1: (Prior Output Preview / Final Decoder Output)
Here is the ipynb file i tested: Same results on both ARC and GPU Max.
Preview modules are taken from here: https://huggingface.co/spaces/multimodalart/stable-cascade/tree/main/previewer
{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"id": "0a186e03-0171-41b5-8d50-13cba3333e41",
"metadata": {},
"outputs": [],
"source": [
"#pip install --force-reinstall torch==2.1.0a0 torchvision==0.16.0a0 intel-extension-for-pytorch==2.1.10+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e254e5fc-8857-4528-89be-69813e177b36",
"metadata": {},
"outputs": [],
"source": [
"#pip install --force-reinstall tensorboard==2.14.1 tensorflow==2.14.0 intel-extension-for-tensorflow[xpu]==2.14.0.1"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2e073708-9e78-4149-a441-7c46bf10b67a",
"metadata": {},
"outputs": [],
"source": [
"#pip install git+https://github.com/kashif/diffusers.git@wuerstchen-v3 accelerate transformers typing_extensions"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1873010a-0a0c-4c1c-bc08-02c87f3f39d1",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"import torch\n",
"import intel_extension_for_pytorch"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "af18ccef-8328-4383-839f-10df6d2d73e0",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"from diffusers import StableCascadeDecoderPipeline, StableCascadePriorPipeline\n",
"prior = StableCascadePriorPipeline.from_pretrained(\"stabilityai/stable-cascade-prior\", torch_dtype=torch.bfloat16).to(\"xpu\")\n",
"decoder = StableCascadeDecoderPipeline.from_pretrained(\"stabilityai/stable-cascade\", torch_dtype=torch.bfloat16).to(\"xpu\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "38fd9e08-3ad3-4efe-b271-c70865517373",
"metadata": {},
"outputs": [],
"source": [
"from IPython.display import clear_output\n",
"from diffusers.utils import numpy_to_pil\n",
"from previewer import Previewer\n",
"previewer = Previewer()\n",
"previewer_state_dict = torch.load(\"previewer_v1_100k.pt\", map_location=torch.device('cpu'))[\"state_dict\"]\n",
"previewer.load_state_dict(previewer_state_dict)\n",
"previewer = previewer.eval().requires_grad_(False).to(\"xpu\", dtype=torch.bfloat16)\n",
"def callback_prior(i, t, latents):\n",
" output = previewer(latents)\n",
" output = numpy_to_pil(output.clamp(0, 1).permute(0, 2, 3, 1).float().cpu().numpy())\n",
" clear_output()\n",
" display(output[0])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0ce30cca-6dd0-48bd-8dba-4ef30c94c30f",
"metadata": {},
"outputs": [],
"source": [
"num_images_per_prompt = 1\n",
"callback_steps = 1\n",
"prompt = \"Anthropomorphic cat dressed as a pilot\"\n",
"negative_prompt = \"\"\n",
"\n",
"torch.xpu.empty_cache()\n",
"prior_output = prior(\n",
" prompt=prompt,\n",
" height=1024,\n",
" width=1024,\n",
" negative_prompt=negative_prompt,\n",
" guidance_scale=4.0,\n",
" num_images_per_prompt=num_images_per_prompt,\n",
" num_inference_steps=20,\n",
" callback=callback_prior,\n",
" callback_steps=callback_steps,\n",
")\n",
"torch.xpu.empty_cache()\n",
"decoder_output = decoder(\n",
" image_embeddings=prior_output.image_embeddings,\n",
" prompt=prompt,\n",
" negative_prompt=negative_prompt,\n",
" guidance_scale=0.0,\n",
" output_type=\"pil\",\n",
" num_inference_steps=10\n",
").images\n",
"torch.xpu.empty_cache()\n",
"\n",
"display(decoder_output[0])"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.7"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
alexsin368
qiacheng
simonlui
srinarayan-srikanthan
Describe the bug
Wuerstchen and Wuerstchen based Stable Cascade models generates corrupted images.
Might be related to my old corruption issue (https://github.com/intel/intel-extension-for-pytorch/issues/519) but this one happens with any resolution and happens with GPU Max too.
Example of the corruption:
Wuerstchen: https://huggingface.co/warp-ai/wuerstchen
Stable Cascade: https://huggingface.co/stabilityai/stable-cascade
Versions