Issues to make it work with RX 5700 XT

[Domcaofficial]

Hello i´m having issues to even start.

_> shark_tank local cache is located at C:\Users\Administrator.local/shark_tank/ . You may change this by setting the --local_tank_cache= flag

vulkan devices are available. cuda devices are not available. Running on local URL: http://0.0.0.0:8080

To create a public link, set share=True in launch(). Found device AMD Radeon RX 5700 XT. Using target triple rdna2-unknown-windows. Using tuned models for stabilityai/stable-diffusion-2-1/fp16/vulkan://00000000-1200-0000-0000-000000000000. torch\jit_check.py:172: UserWarning: The TorchScript type system doesn't support instance-level annotations on empty non-base types in __init__. Instead, either 1) use a type annotation in the class body, or 2) wrap the type in torch.jit.Attribute. warnings.warn("The TorchScript type system doesn't support " loading existing vmfb from: C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\euler_scale_model_input_1_512_512fp16.vmfbWARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files. loading existing vmfb from: C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\euler_step_1_512_512fp16.vmfb WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files. Inferring base model configuration. safetensors\torch.py:98: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() torch_utils.py:776: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() return self.fget.get(instance, owner)() torch\storage.py:899: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage() storage = cls(wrap_storage=untyped_storage) Loading Winograd config file from C:\Users\Administrator.local/shark_tank/configs\unet_winograd_vulkan.json 100%|█████████████████████████████████████████████████████████████████████████████████| 107/107 [00:00<00:00, 10.7kB/s] 100%|█████████████████████████████████████████████████████████████████████████████████| 107/107 [00:00<00:00, 10.7kB/s] Loading lowering config file from C:\Users\Administrator.local/shark_tank/configs\unet_v2_1base_fp16_vulkan_rdna2.json100%|██████████████████████████████████████████████████████████████████████████████| 24.2k/24.2k [00:00<00:00, 194kB/s] 100%|█████████████████████████████████████████████████████████████████████████████| 24.2k/24.2k [00:00<00:00, 2.25MB/s] Applying tuned configs on unet1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan-00000000-1200-0000-0000-000000000000 No vmfb found. Compiling and saving to C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\unet1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan-00000000-1200-0000-0000-000000000000.vmfb Using target triple -iree-vulkan-target-triple=rdna2-unknown-windows from command line args Saved vmfb in C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\unet1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan-00000000-1200-0000-0000-000000000000.vmfb. WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files. torch\fx\node.py:250: UserWarning: Trying to prepend a node to itself. This behavior has no effect on the graph. warnings.warn("Trying to prepend a node to itself. This behavior has no effect on the graph.") Loading Winograd config file from C:\Users\Administrator.local/shark_tank/configs\vae_winograd_vulkan.json 100%|█████████████████████████████████████████████████████████████████████████████████| 33.0/33.0 [00:00<00:00, 250B/s] 100%|███████████████████████████████████████████████████████████████████████████████| 33.0/33.0 [00:00<00:00, 2.75kB/s] No vmfb found. Compiling and saving to C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\vae1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan-00000000-1200-0000-0000-000000000000.vmfb Using target triple -iree-vulkan-target-triple=rdna2-unknown-windows from command line args Saved vmfb in C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\vae1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan-00000000-1200-0000-0000-000000000000.vmfb. WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files. Retrying with a different base model configuration Retrying with a different base model configuration Retrying with a different base model configuration Retrying with a different base model configuration Traceback (most recent call last): File "gradio\routes.py", line 384, in run_predict File "gradio\blocks.py", line 1024, in process_api File "gradio\blocks.py", line 850, in call_function File "anyio\to_thread.py", line 31, in run_sync File "anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread File "anyio_backends_asyncio.py", line 867, in run File "gradio\utils.py", line 445, in async_iteration File "apps\stable_diffusion\scripts\txt2img.py", line 117, in txt2img_inf File "apps\stable_diffusion\src\pipelines\pipeline_shark_stable_diffusion_utils.py", line 355, in from_pretrained File "apps\stable_diffusion\src\models\modelwrappers.py", line 541, in call SystemExit: Cannot compile the model. Please create an issue with the detailed log at https://github.com/nod-ai/SHARK/issues

Thank you.

[Domcaofficial]

I have tried every version from 573 to 577 and none of those worked

[yzhang93]

@Domcaofficial Looks like the models are already compiled which is good. Is this your first attempt of SHARK? Can you try append the flag --clear_all?

[Domcaofficial]

Yes its my first attempt of Shark,

[Domcaofficial]

Tried again, at one point gpu tried to generate something, but result was error

1. C:\Users\Administrator\Desktop\AI IMAGE GENERATOR>shark_sd_20230301_573.exe --no-use_tuned

• shark_tank local cache is located at C:\Users\Administrator.local/shark_tank/ . You may change this by setting the --local_tank_cache= flag
• vulkan devices are available.
• cuda devices are not available.
• gradio\components.py:148: UserWarning: Unknown style parameter: full_width
• Running on local URL: http://0.0.0.0:8080
• To create a public link, set share=True in launch().
• Found device AMD Radeon RX 5700 XT. Using target triple rdna2-unknown-windows.
• Using tuned models for stabilityai/stable-diffusion-2-1/fp16/vulkan://00000000-1200-0000-0000-000000000000.
• torch\jit_check.py:172: UserWarning: The TorchScript type system doesn't support instance-level annotations on empty non-base types in __init__. Instead, either 1) use a type annotation in the class body, or 2) wrap the type in torch.jit.Attribute.
• warnings.warn("The TorchScript type system doesn't support "
• No vmfb found. Compiling and saving to C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\euler_scale_model_input_1_512_512fp16.vmfb
• Using target triple -iree-vulkan-target-triple=rdna2-unknown-windows from command line args
• Saved vmfb in C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\euler_scale_model_input_1_512_512fp16.vmfb.
• WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
• No vmfb found. Compiling and saving to C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\euler_step_1_512_512fp16.vmfb
• Using target triple -iree-vulkan-target-triple=rdna2-unknown-windows from command line args
• Saved vmfb in C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\euler_step_1_512_512fp16.vmfb.
• WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
• Inferring base model configuration.
• safetensors\torch.py:98: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
• torch_utils.py:776: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
• return self.fget.get(instance, owner)()
• torch\storage.py:899: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
• storage = cls(wrap_storage=untyped_storage)
• torch\fx\node.py:250: UserWarning: Trying to prepend a node to itself. This behavior has no effect on the graph.
• warnings.warn("Trying to prepend a node to itself. This behavior has no effect on the graph.")
• Loading Winograd config file from C:\Users\Administrator.local/shark_tank/configs\unet_winograd_vulkan.json
• 100%|█████████████████████████████████████████████████████████████████████████████████| 107/107 [00:00<00:00, 10.7kB/s]
• 100%|█████████████████████████████████████████████████████████████████████████████████| 107/107 [00:00<00:00, 10.7kB/s]
• Loading lowering config file from C:\Users\Administrator.local/shark_tank/configs\unet_v2_1base_fp16_vulkan_rdna2.json100%|█████████████████████████████████████████████████████████████████████████████| 24.2k/24.2k [00:00<00:00, 1.46MB/s]
• 100%|█████████████████████████████████████████████████████████████████████████████| 24.2k/24.2k [00:00<00:00, 1.46MB/s]
• Applying tuned configs on unet1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan-00000000-1200-0000-0000-000000000000
• No vmfb found. Compiling and saving to C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\unet1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan-00000000-1200-0000-0000-000000000000.vmfb
• Using target triple -iree-vulkan-target-triple=rdna2-unknown-windows from command line args
• Saved vmfb in C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\unet1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan-00000000-1200-0000-0000-000000000000.vmfb.
• WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
• Loading Winograd config file from C:\Users\Administrator.local/shark_tank/configs\vae_winograd_vulkan.json
• 100%|███████████████████████████████████████████████████████████████████████████████████████| 33.0/33.0 [00:00<00:00, 4.12kB/s]
• 100%|███████████████████████████████████████████████████████████████████████████████████████| 33.0/33.0 [00:00<00:00, 3.30kB/s]
• No vmfb found. Compiling and saving to C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\vae1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan-00000000-1200-0000-0000-000000000000.vmfb
• Using target triple -iree-vulkan-target-triple=rdna2-unknown-windows from command line args
• Saved vmfb in C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\vae1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan-00000000-1200-0000-0000-000000000000.vmfb.
• WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
• Downloading (…)_encoder/config.json: 100%|█████████████████████████████████████████████████████| 613/613 [00:00<00:00, 613kB/s]
• Downloading (…)"model.safetensors";: 100%|████████████████████████████████████████████████| 1.36G/1.36G [00:49<00:00, 27.6MB/s]
• transformers\modeling_utils.py:402: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
• No vmfb found. Compiling and saving to C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\clip1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan-00000000-1200-0000-0000-000000000000.vmfb
• Using target triple -iree-vulkan-target-triple=rdna2-unknown-windows from command line args
• Saved vmfb in C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\clip1_64_512_512_fp16_tuned_stable-diffusion-2-1-base_vulkan-00000000-1200-0000-0000-000000000000.vmfb.
• WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
• Downloading (…)tokenizer/vocab.json: 100%|████████████████████████████████████████████████| 1.06M/1.06M [00:00<00:00, 1.44MB/s]
• Downloading (…)tokenizer/merges.txt: 100%|███████████████████████████████████████████████████| 525k/525k [00:01<00:00, 430kB/s]
• Downloading (…)cial_tokens_map.json: 100%|█████████████████████████████████████████████████████| 460/460 [00:00<00:00, 461kB/s]
• Downloading (…)okenizer_config.json: 100%|█████████████████████████████████████████████████████| 824/824 [00:00<00:00, 823kB/s]
• 50it [00:11, 4.26it/s]
• 0%| | 0/1 [00:00<?, ?it/s]
• Traceback (most recent call last):
• File "gradio\routes.py", line 384, in run_predict
• File "gradio\blocks.py", line 1024, in process_api
• File "gradio\blocks.py", line 850, in call_function
• File "anyio\to_thread.py", line 31, in run_sync
• File "anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread
• File "anyio_backends_asyncio.py", line 867, in run
• File "gradio\utils.py", line 445, in async_iteration
• File "apps\stable_diffusion\scripts\txt2img.py", line 143, in txt2img_inf
• File "apps\stable_diffusion\src\pipelines\pipeline_shark_stable_diffusion_txt2img.py", line 132, in generate_images
• File "apps\stable_diffusion\src\pipelines\pipeline_shark_stable_diffusion_utils.py", line 99, in decode_latents
• File "shark\shark_inference.py", line 138, in call
• File "shark\shark_runner.py", line 93, in run
• File "shark\iree_utils\compile_utils.py", line 381, in get_results
• File "iree\runtime\function.py", line 130, in call
• File "iree\runtime\function.py", line 154, in _invoke
• RuntimeError: Error invoking function: D:\a\SHARK-Runtime\SHARK-Runtime\c\runtime\src\iree\hal\drivers\vulkan\vma_allocator.cc:693: RESOURCE_EXHAUSTED; VK_ERROR_OUT_OF_DEVICE_MEMORY; vmaCreateBuffer; while invoking native function hal.device.queue.alloca; while calling import;
• [ 1] native hal.device.queue.alloca:0 -
• [ 0] bytecode module.forward:3150 [
• .13:8:18,
• .13:11:20,
• .13:13:27,
• .13:14:15,
• .13:22:12,
• .13:41:20,
• .13:38:11,
• .13:43:29,
• .13:44:17,
• .13:52:12,
• .13:68:13,
• .13:71:20,
• .13:72:12,
• .13:75:29,
• .13:76:17,
• .13:84:12,
• .13:95:12,
• .13:101:16,
• .13:118:12,
• .13:143:18,
• .13:111:12,
• .13:144:14,
• .13:145:15,
• .13:131:12,
• .13:147:12,
• .13:155:12,
• .13:159:13,
• .13:163:29,
• .13:164:17,
• .13:172:12,
• .13:188:13,
• .13:191:20,
• .13:193:29,
• .13:194:17,
• .13:202:12,
• .13:218:13,
• .13:221:20,
• .13:222:13,
• .13:226:29,
• .13:227:17,
• .13:235:13,
• .13:251:13,
• .13:254:20,
• .13:256:29,
• .13:257:17,
• .13:265:13,
• .13:281:13,
• .13:284:20,
• .13:285:13,
• .13:289:29,
• .13:290:17,
• .13:298:13,
• .13:314:13,
• .13:317:20,
• .13:319:29,
• .13:320:17,
• .13:328:13,
• .13:344:13,
• .13:347:20,
• .13:348:13,
• .13:352:29,
• .13:353:17,
• .13:361:13,
• .13:377:13,
• .13:380:21,
• .13:382:30,
• .13:383:18,
• .13:391:13,
• .13:407:13,
• .13:410:21,
• .13:411:13,
• .13:416:21,
• .13:413:25,
• .13:418:30,
• .13:419:18,
• .13:427:13,
• .13:443:14,
• .13:446:21,
• .13:448:30,
• .13:449:18,
• .13:457:13,
• .13:473:14,
• .13:476:21,
• .13:477:13,
• .13:480:30,
• .13:481:18,
• .13:489:13,
• .13:505:14,
• .13:508:21,
• .13:510:30,
• .13:511:18,
• .13:519:13,
• .13:535:14,
• .13:538:21,
• .13:539:13,
• .13:542:30,
• .13:543:18,
• .13:551:13,
• .13:567:14,
• .13:570:21,
• .13:572:30,
• .13:573:18,
• .13:581:13,
• .13:597:14,
• .13:600:21,
• .13:601:13,
• .13:606:21,
• .13:603:27,
• .13:608:30,
• .13:609:18,
• .13:617:13,
• .13:633:14,
• .13:636:21,
• .13:638:30,
• .13:639:18,
• .13:647:13,
• .13:666:21,
• .13:663:14,
• .13:669:21,
• .13:670:13,
• .13:673:30,
• .13:674:18,
• .13:682:13,
• .13:698:14,
• .13:701:21,
• .13:703:30,
• .13:704:18,
• .13:712:13,
• .13:728:14,
• .13:731:21,
• .13:732:13,
• .13:735:30,
• .13:736:18,
• .13:744:13,
• .13:760:14,
• .13:763:21,
• .13:765:30,
• .13:766:18,
• .13:774:13,
• .13:790:14,
• .13:793:21,
• .13:794:13,
• .13:799:21,
• .13:796:27,
• .13:801:30,
• .13:802:18,
• .13:810:13,
• .13:826:14,
• .13:829:21,
• .13:831:30,
• .13:832:18,
• .13:840:13,
• .13:859:21,
• .13:856:14,
• .13:862:21,
• .13:863:13,
• .13:866:30,
• .13:867:18,
• .13:875:13,
• .13:891:14,
• .13:894:21,
• .13:896:30,
• .13:897:18,
• .13:905:13,
• .13:921:14,
• .13:924:21,
• .13:925:13,
• .13:928:30,
• .13:929:18,
• .13:937:13,
• .13:953:14,
• .13:956:21,
• .13:958:30,
• .13:959:18,
• .13:967:13,
• .13:983:14,
• .13:986:21,
• .13:987:13,
• .13:990:30,
• .13:991:18,
• .13:999:13,
• .13:1015:14,
• .13:1018:21
• ]
• 50it [00:11, 4.24it/s]
• 0%| | 0/1 [00:00<?, ?it/s]
• Traceback (most recent call last):
• File "gradio\routes.py", line 384, in run_predict
• File "gradio\blocks.py", line 1024, in process_api
• File "gradio\blocks.py", line 850, in call_function
• File "anyio\to_thread.py", line 31, in run_sync
• File "anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread
• File "anyio_backends_asyncio.py", line 867, in run
• File "gradio\utils.py", line 445, in async_iteration
• File "apps\stable_diffusion\scripts\txt2img.py", line 143, in txt2img_inf
• File "apps\stable_diffusion\src\pipelines\pipeline_shark_stable_diffusion_txt2img.py", line 132, in generate_images
• File "apps\stable_diffusion\src\pipelines\pipeline_shark_stable_diffusion_utils.py", line 99, in decode_latents
• File "shark\shark_inference.py", line 138, in call
• File "shark\shark_runner.py", line 93, in run
• File "shark\iree_utils\compile_utils.py", line 381, in get_results
• File "iree\runtime\function.py", line 130, in call
• File "iree\runtime\function.py", line 154, in _invoke
• RuntimeError: Error invoking function: D:\a\SHARK-Runtime\SHARK-Runtime\c\runtime\src\iree\hal\drivers\vulkan\vma_allocator.cc:693: RESOURCE_EXHAUSTED; VK_ERROR_OUT_OF_DEVICE_MEMORY; vmaCreateBuffer; while invoking native function hal.device.queue.alloca; while calling import;
• [ 1] native hal.device.queue.alloca:0 -
• [ 0] bytecode module.forward:3150 [
• .13:8:18,
• .13:11:20,
• .13:13:27,
• .13:14:15,
• .13:22:12,
• .13:41:20,
• .13:38:11,
• .13:43:29,
• .13:44:17,
• .13:52:12,
• .13:68:13,
• .13:71:20,
• .13:72:12,
• .13:75:29,
• .13:76:17,
• .13:84:12,
• .13:95:12,
• .13:101:16,
• .13:118:12,
• .13:143:18,
• .13:111:12,
• .13:144:14,
• .13:145:15,
• .13:131:12,
• .13:147:12,
• .13:155:12,
• .13:159:13,
• .13:163:29,
• .13:164:17,
• .13:172:12,
• .13:188:13,
• .13:191:20,
• .13:193:29,
• .13:194:17,
• .13:202:12,
• .13:218:13,
• .13:221:20,
• .13:222:13,
• .13:226:29,
• .13:227:17,
• .13:235:13,
• .13:251:13,
• .13:254:20,
• .13:256:29,
• .13:257:17,
• .13:265:13,
• .13:281:13,
• .13:284:20,
• .13:285:13,
• .13:289:29,
• .13:290:17,
• .13:298:13,
• .13:314:13,
• .13:317:20,
• .13:319:29,
• .13:320:17,
• .13:328:13,
• .13:344:13,
• .13:347:20,
• .13:348:13,
• .13:352:29,
• .13:353:17,
• .13:361:13,
• .13:377:13,
• .13:380:21,
• .13:382:30,
• .13:383:18,
• .13:391:13,
• .13:407:13,
• .13:410:21,
• .13:411:13,
• .13:416:21,
• .13:413:25,
• .13:418:30,
• .13:419:18,
• .13:427:13,
• .13:443:14,
• .13:446:21,
• .13:448:30,
• .13:449:18,
• .13:457:13,
• .13:473:14,
• .13:476:21,
• .13:477:13,
• .13:480:30,
• .13:481:18,
• .13:489:13,
• .13:505:14,
• .13:508:21,
• .13:510:30,
• .13:511:18,
• .13:519:13,
• .13:535:14,
• .13:538:21,
• .13:539:13,
• .13:542:30,
• .13:543:18,
• .13:551:13,
• .13:567:14,
• .13:570:21,
• .13:572:30,
• .13:573:18,
• .13:581:13,
• .13:597:14,
• .13:600:21,
• .13:601:13,
• .13:606:21,
• .13:603:27,
• .13:608:30,
• .13:609:18,
• .13:617:13,
• .13:633:14,
• .13:636:21,
• .13:638:30,
• .13:639:18,
• .13:647:13,
• .13:666:21,
• .13:663:14,
• .13:669:21,
• .13:670:13,
• .13:673:30,
• .13:674:18,
• .13:682:13,
• .13:698:14,
• .13:701:21,
• .13:703:30,
• .13:704:18,
• .13:712:13,
• .13:728:14,
• .13:731:21,
• .13:732:13,
• .13:735:30,
• .13:736:18,
• .13:744:13,
• .13:760:14,
• .13:763:21,
• .13:765:30,
• .13:766:18,
• .13:774:13,
• .13:790:14,
• .13:793:21,
• .13:794:13,
• .13:799:21,
• .13:796:27,
• .13:801:30,
• .13:802:18,
• .13:810:13,
• .13:826:14,
• .13:829:21,
• .13:831:30,
• .13:832:18,
• .13:840:13,
• .13:859:21,
• .13:856:14,
• .13:862:21,
• .13:863:13,
• .13:866:30,
• .13:867:18,
• .13:875:13,
• .13:891:14,
• .13:894:21,
• .13:896:30,
• .13:897:18,
• .13:905:13,
• .13:921:14,
• .13:924:21,
• .13:925:13,
• .13:928:30,
• .13:929:18,
• .13:937:13,
• .13:953:14,
• .13:956:21,
• .13:958:30,
• .13:959:18,
• .13:967:13,
• .13:983:14,
• .13:986:21,
• .13:987:13,
• .13:990:30,
• .13:991:18,
• .13:999:13,
• .13:1015:14,
• .13:1018:21
• ]
• Found device AMD Radeon RX 5700 XT. Using target triple rdna2-unknown-windows.
• Using tuned models for stabilityai/stable-diffusion-2-1-base/fp16/vulkan://00000000-1200-0000-0000-000000000000.
• loading existing vmfb from: C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\euler_scale_model_input_1_512_512fp16.vmfb
• WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
• loading existing vmfb from: C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\euler_step_1_512_512fp16.vmfb
• WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
• WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
• WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
• Traceback (most recent call last):
• File "gradio\routes.py", line 384, in run_predict
• File "gradio\blocks.py", line 1024, in process_api
• File "gradio\blocks.py", line 850, in call_function
• File "anyio\to_thread.py", line 31, in run_sync
• File "anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread
• File "anyio_backends_asyncio.py", line 867, in run
• File "gradio\utils.py", line 445, in async_iteration
• File "apps\stable_diffusion\scripts\txt2img.py", line 117, in txt2img_inf
• File "apps\stable_diffusion\src\pipelines\pipeline_shark_stable_diffusion_utils.py", line 232, in from_pretrained
• File "apps\stable_diffusion\src\models\model_wrappers.py", line 333, in call
• File "apps\stable_diffusion\src\utils\utils.py", line 488, in fetch_or_delete_vmfbs
• File "apps\stable_diffusion\src\utils\utils.py", line 456, in load_vmfb
• File "shark\shark_inference.py", line 207, in load_module
• File "shark\iree_utils\compile_utils.py", line 329, in load_flatbuffer
• File "shark\iree_utils\compile_utils.py", line 305, in get_iree_module
• File "iree\runtime\system_api.py", line 255, in add_vm_module
• File "iree\runtime\system_api.py", line 252, in add_vm_modules
• RuntimeError: Error registering modules: D:\a\SHARK-Runtime\SHARK-Runtime\c\runtime\src\iree\hal\drivers\vulkan\vma_allocator.cc:693: RESOURCE_EXHAUSTED; VK_ERROR_OUT_OF_DEVICE_MEMORY; vmaCreateBuffer; while invoking native function hal.allocator.allocate; while calling import;
• [ 1] native hal.allocator.allocate:0 -
• [ 0] bytecode module.__init:27660 [
• .9:26:8,
• .9:31:10,
• .9:69:10,
• .9:162:10,
• .9:167:10,
• .9:172:10,
• .9:200:10,
• .9:221:10,
• .9:226:10,
• .9:231:11,
• .9:259:11,
• .9:280:11,
• .9:290:11,
• .9:297:11,
• .9:338:11,
• .9:431:11,
• .9:436:11,
• .9:441:11,
• .9:469:11,
• .9:490:11,
• .9:495:11,
• .9:500:11,
• .9:528:11,
• .9:549:11,
• .9:559:11,
• .9:566:11,
• .9:610:11,
• .9:706:11,
• .9:711:11,
• .9:716:11,
• .9:744:11,
• .9:765:11,
• .9:770:11,
• .9:775:11,
• .9:803:11,
• .9:824:11,
• .9:834:11,
• .9:841:11,
• .9:882:11,
• .9:975:11,
• .9:980:11,
• .9:985:11,
• .9:1013:11,
• .9:1034:11,
• .9:1039:11,
• .9:1044:11,
• .9:1072:11,
• .9:1093:11,
• .9:1103:11,
• .9:1110:11,
• .9:1154:11,
• .9:1250:11,
• .9:1255:11,
• .9:1260:11,
• .9:1288:11,
• .9:1309:11,
• .9:1314:11,
• .9:1319:11,
• .9:1347:11,
• .9:1368:11,
• .9:1378:11,
• .9:1385:11,
• .9:1426:11,
• .9:1519:11,
• .9:1524:11,
• .9:1529:11,
• .9:1557:11,
• .9:1578:11,
• .9:1583:11,
• .9:1588:11,
• .9:1616:11,
• .9:1637:11,
• .9:1647:11,
• .9:1654:11,
• .9:1698:11,
• .9:1770:11,
• .9:1842:11,
• .9:1935:11,
• .9:1940:11,
• .9:1945:11,
• .9:1973:11,
• .9:1994:11,
• .9:1999:11,
• .9:2004:11,
• .9:2032:11,
• .9:2053:11,
• .9:2063:11,
• .9:2070:11,
• .9:2111:11,
• .9:2184:11,
• .9:2260:11,
• .9:2336:11,
• .9:2416:11,
• .9:2512:12,
• .9:2517:12,
• .9:2522:12,
• .9:2550:12,
• .9:2571:12,
• .9:2576:12,
• .9:2581:12,
• .9:2609:12,
• .9:2630:12,
• .9:2640:12,
• .9:2647:12,
• .9:2689:12,
• .9:2785:12,
• .9:2790:12,
• .9:2795:12,
• .9:2823:12,
• .9:2844:12,
• .9:2849:12,
• .9:2854:12,
• .9:2882:12,
• .9:2903:12,
• .9:2913:12,
• .9:2920:12,
• .9:2962:12,
• .9:3058:12,
• .9:3063:12,
• .9:3068:12,
• .9:3096:12,
• .9:3117:12,
• .9:3122:12,
• .9:3127:12,
• .9:3155:12,
• .9:3176:12,
• .9:3186:12,
• .9:3193:12,
• .9:3239:12,
• .9:3335:12,
• .9:3340:12,
• .9:3345:12,
• .9:3373:12,
• .9:3394:12,
• .9:3399:12,
• .9:3404:12,
• .9:3432:12,
• .9:3453:12,
• .9:3463:12,
• .9:3470:12,
• .9:3512:12,
• .9:3608:12,
• .9:3613:12,
• .9:3618:12,
• .9:3646:12,
• .9:3667:12,
• .9:3672:12,
• .9:3677:12,
• .9:3705:12,
• .9:3726:12,
• .9:3736:12,
• .9:3743:12,
• .9:3785:12,
• .9:3881:12,
• .9:3886:12,
• .9:3891:12,
• .9:3919:12,
• .9:3940:12,
• .9:3945:12,
• .9:3950:12,
• .9:3978:12,
• .9:3999:12,
• .9:4009:12,
• .9:4016:12,
• .9:4062:12,
• .9:4158:12,
• .9:4163:12,
• .9:4168:12,
• .9:4196:12,
• .9:4217:12,
• .9:4222:12,
• .9:4227:12,
• .9:4255:12,
• .9:4276:12,
• .9:4286:12,
• .9:4293:12,
• .9:4335:12,
• .9:4431:12,
• .9:4436:12,
• .9:4441:12,
• .9:4469:12,
• .9:4490:12,
• .9:4495:12,
• .9:4500:12,
• .9:4528:12,
• .9:4549:12,
• .9:4559:12,
• .9:4566:12,
• .9:4608:12,
• .9:4704:12,
• .9:4709:12,
• .9:4714:12,
• .9:4742:12,
• .9:4763:12,
• .9:4768:12,
• .9:4773:12,
• .9:4801:12,
• .9:4822:12,
• .9:4832:12,
• .9:4839:12,
• .9:57:12,
• .9:58:12,
• .9:97:12,
• .9:98:12,
• .9:129:12,
• .9:130:12,
• .9:326:13,
• .9:327:13,
• .9:366:13,
• .9:367:13,
• .9:398:13,
• .9:399:13,
• .9:598:13,
• .9:599:13,
• .9:638:13,
• .9:639:13,
• .9:673:13,
• .9:674:13,
• .9:870:13,
• .9:871:13,
• .9:910:13,
• .9:911:13,
• .9:942:13,
• .9:943:13,
• .9:1142:13,
• .9:1143:13,
• .9:1182:13,
• .9:1183:13,
• .9:1217:13,
• .9:1218:13,
• .9:1414:13,
• .9:1415:13,
• .9:1454:13,
• .9:1455:13,
• .9:1486:13,
• .9:1487:14,
• .9:1686:13,
• .9:1687:14,
• .9:1726:13,
• .9:1727:14,
• .9:1758:13,
• .9:1759:14,
• .9:1798:13,
• .9:1799:14,
• .9:1830:13,
• .9:1831:14,
• .9:1870:13,
• .9:1871:14,
• .9:1902:13,
• .9:1903:14,
• .9:2099:14,
• .9:2100:14,
• .9:2139:14,
• .9:2140:14,
• .9:2172:14,
• .9:2173:14,
• .9:2212:14,
• .9:2213:14,
• .9:2248:14,
• .9:2249:14,
• .9:2288:14,
• .9:2289:14,
• .9:2324:14,
• .9:2325:14,
• .9:2364:14,
• .9:2365:14,
• .9:2404:14,
• .9:2405:14,
• .9:2444:14,
• .9:2445:14,
• .9:2479:14,
• .9:2480:14,
• .9:2677:14,
• .9:2678:14,
• .9:2717:14,
• .9:2718:14,
• .9:2752:14,
• .9:2753:14,
• .9:2950:14,
• .9:2951:14,
• .9:2990:14,
• .9:2991:14,
• .9:3025:14,
• .9:3026:14,
• .9:3227:14,
• .9:3228:14,
• .9:3267:14,
• .9:3268:14,
• .9:3302:14,
• .9:3303:14,
• .9:3500:14,
• .9:3501:14,
• .9:3540:14,
• .9:3541:14,
• .9:3575:14,
• .9:3576:14,
• .9:3773:14,
• .9:3774:14,
• .9:3813:14,
• .9:3814:14,
• .9:3848:14,
• .9:3849:14,
• .9:4050:14,
• .9:4051:14,
• .9:4090:14,
• .9:4091:14,
• .9:4125:14,
• .9:4126:14,
• .9:4323:14,
• .9:4324:14,
• .9:4363:14,
• .9:4364:14,
• .9:4398:14,
• .9:4399:14,
• .9:4596:14,
• .9:4597:14,
• .9:4636:14,
• .9:4637:14,
• .9:4671:14,
• .9:4672:14,
• .9:4868:14,
• .9:4869:14,
• .9:36:18,
• .9:577:20,
• .9:650:20,
• .9:1121:21,
• .9:1191:21,
• .9:1194:21,
• .9:1423:21,
• .9:1463:21,
• .9:1665:21,
• .9:1695:21,
• .9:1735:21,
• .9:1767:21,
• .9:1807:21,
• .9:1839:21,
• .9:1879:21,
• .9:2108:21,
• .9:2148:21,
• .9:2181:21,
• .9:2221:21,
• .9:2224:21,
• .9:2257:21,
• .9:2297:21,
• .9:2300:21,
• .9:2333:21,
• .9:2373:21,
• .9:2376:21,
• .9:2382:21,
• .9:2413:21,
• .9:2453:21,
• .9:2456:21,
• .9:2686:21,
• .9:2726:21,
• .9:2729:21,
• .9:2959:21,
• .9:2999:21,
• .9:3002:21,
• .9:3205:21,
• .9:3236:21,
• .9:3279:21,
• .9:3552:21,
• .9:3825:21,
• .9:4102:21,
• .9:4375:21,
• .9:4648:21,
• .9:4877:21
• ]
• Found device AMD Radeon RX 5700 XT. Using target triple rdna2-unknown-windows.
• Using tuned models for CompVis/stable-diffusion-v1-4/fp16/vulkan://00000000-1200-0000-0000-000000000000.
• loading existing vmfb from: C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\euler_scale_model_input_1_512_512fp16.vmfb
• WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
• loading existing vmfb from: C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\euler_step_1_512_512fp16.vmfb
• WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
• Inferring base model configuration.
• Downloading (…)_model.safetensors";: 100%|████████████████████████████████████████████████| 3.44G/3.44G [02:39<00:00, 21.6MB/s]
• Downloading (…)ain/unet/config.json: 100%|█████████████████████████████████████████████████████| 743/743 [00:00<00:00, 743kB/s]
• Retrying with a different base model configuration
• Loading Winograd config file from C:\Users\Administrator.local/shark_tank/configs\vae_winograd_vulkan.json
• 100%|███████████████████████████████████████████████████████████████████████████████████████| 33.0/33.0 [00:00<00:00, 2.75kB/s]
• 100%|███████████████████████████████████████████████████████████████████████████████████████| 33.0/33.0 [00:00<00:00, 2.36kB/s]
• No vmfb found. Compiling and saving to C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\unet1_64_512_512_fp16_tuned_stable-diffusion-v1-4_vulkan-00000000-1200-0000-0000-000000000000.vmfb
• Using target triple -iree-vulkan-target-triple=rdna2-unknown-windows from command line args
• Saved vmfb in C:\Users\Administrator\Desktop\AI IMAGE GENERATOR\unet1_64_512_512_fp16_tuned_stable-diffusion-v1-4_vulkan-00000000-1200-0000-0000-000000000000.vmfb.
• WARNING: [Loader Message] Code 0 : windows_read_data_files_in_registry: Registry lookup failed to get layer manifest files.
• Retrying with a different base model configuration
• Retrying with a different base model configuration
• Retrying with a different base model configuration
• Traceback (most recent call last):
• File "gradio\routes.py", line 384, in run_predict
• File "gradio\blocks.py", line 1024, in process_api
• File "gradio\blocks.py", line 850, in call_function
• File "anyio\to_thread.py", line 31, in run_sync
• File "anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread
• File "anyio_backends_asyncio.py", line 867, in run
• File "gradio\utils.py", line 445, in async_iteration
• File "apps\stable_diffusion\scripts\txt2img.py", line 117, in txt2img_inf
• File "apps\stable_diffusion\src\pipelines\pipeline_shark_stable_diffusion_utils.py", line 232, in from_pretrained
• File "apps\stable_diffusion\src\models\model_wrappers.py", line 398, in call
• SystemExit: Cannot compile the model. Please create an issue with the detailed log at https://github.com/nod-ai/SHARK/issues

[yzhang93]

Shark package 573-577 was broken, can you try our new release 578 and let us know if it works.

[RoyalflushLiu]

I think this happens due to memory exhaustion

• RuntimeError: Error invoking function: D:\a\SHARK-Runtime\SHARK-Runtime\c\runtime\src\iree\hal\drivers\vulkan\vma_allocator.cc:693: RESOURCE_EXHAUSTED; VK_ERROR_OUT_OF_DEVICE_MEMORY; vmaCreateBuffer; while invoking native function hal.device.queue.alloca; while calling import;

Try to append the args: --vulkan_large_heap_block_size=0 --use_base_vae