NVIDIA / Stable-Diffusion-WebUI-TensorRT

TensorRT Extension for Stable Diffusion Web UI
MIT License
1.89k stars 143 forks source link

It stop here. #184

Open BasicGearwheel opened 9 months ago

BasicGearwheel commented 9 months ago

unknown_2023 12 03-12 18_1

Just like this, the training process stopped.My graphics card/GPU is P104-100.

BasicGearwheel commented 9 months ago

venv "F:\stable-diffusion-webui\venv\Scripts\Python.exe" Python 3.10.9 (tags/v3.10.9:1dd9be6, Dec 6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)] Version: v1.6.0-2-g4afaaf8a Commit hash: 4afaaf8a020c1df457bcf7250cb1c7f609699fa7 loading WD14-tagger reqs from F:\stable-diffusion-webui\extensions\stable-diffusion-webui-wd14-tagger\requirements.txt Checking WD14-tagger requirements. Launching Web UI with arguments: --autolaunch --medvram-sdxl --xformers --additional-device-ids cpu:0 2023-12-03 12:14:52.629190: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable TF_ENABLE_ONEDNN_OPTS=0. WARNING:tensorflow:From F:\stable-diffusion-webui\venv\lib\site-packages\keras\src\losses.py:2976: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.

[AddNet] Updating model hashes... 0it [00:00, ?it/s] [AddNet] Updating model hashes... 0it [00:00, ?it/s] dirname: F:\stable-diffusion-webui\localizations localizations: {'zh-Hans (Stable) [vladmandic]': 'F:\stable-diffusion-webui\extensions\stable-diffusion-webui-localization-zh_Hans\localizations\zh-Hans (Stable) [vladmandic].json', 'zh-Hans (Stable)': 'F:\stable-diffusion-webui\extensions\stable-diffusion-webui-localization-zh_Hans\localizations\zh-Hans (Stable).json', 'zh-Hans (Testing) [vladmandic]': 'F:\stable-diffusion-webui\extensions\stable-diffusion-webui-localization-zh_Hans\localizations\zh-Hans (Testing) [vladmandic].json', 'zh-Hans (Testing)': 'F:\stable-diffusion-webui\extensions\stable-diffusion-webui-localization-zh_Hans\localizations\zh-Hans (Testing).json'} 2023-12-03 12:15:30,912 - ControlNet - INFO - ControlNet v1.1.409 ControlNet preprocessor location: F:\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads 2023-12-03 12:15:31,272 - ControlNet - INFO - ControlNet v1.1.409 == WD14 tagger /cpu:0, uname_result(system='Windows', node='DESKTOP-F7IDF0A', release='10', version='10.0.19045', machine='AMD64') == Tag Autocomplete: Could not locate model-keyword extension, Lora trigger word completion will be limited to those added through the extra networks menu. Loading weights [cc6cb27103] from F:\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned-emaonly.ckpt Creating model from config: F:\stable-diffusion-webui\configs\v1-inference.yaml Applying attention optimization: xformers... done. Model loaded in 18.6s (load weights from disk: 6.2s, create model: 1.4s, apply weights to model: 10.1s, apply half(): 0.3s, move model to device: 0.1s, load textual inversion embeddings: 0.2s, calculate empty prompt: 0.3s). F:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\metadata_editor.py:399: GradioDeprecationWarning: The style method is deprecated. Please set these arguments in the constructor instead. with gr.Row().style(equal_height=False): F:\stable-diffusion-webui\extensions\sd-webui-additional-networks\scripts\metadata_editor.py:521: GradioDeprecationWarning: The style method is deprecated. Please set these arguments in the constructor instead. cover_image = gr.Image( Scanning <DirEntry 'wd14-convnext.v2'> as onnx model Scanning <DirEntry 'wd14-swinv2-v1'> as onnx model Scanning <DirEntry 'wd14-vit.v2'> as onnx model Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). [Lobe]: Initializing Lobe Startup time: 87.4s (prepare environment: 15.4s, import torch: 28.7s, import gradio: 2.1s, setup paths: 2.0s, initialize shared: 8.6s, other imports: 1.4s, setup codeformer: 0.3s, load scripts: 7.5s, create ui: 20.4s, gradio launch: 0.9s, add APIs: 0.1s). FP16 has been disabled because your GPU does not support it. Exporting v1-5-pruned-emaonly to TensorRT {'sample': [(1, 4, 64, 64), (2, 4, 64, 64), (8, 4, 96, 96)], 'timesteps': [(1,), (2,), (8,)], 'encoder_hidden_states': [(1, 77, 768), (2, 77, 768), (8, 154, 768)]} Building TensorRT engine... This can take a while, please check the progress in the terminal. Building TensorRT engine for F:\stable-diffusion-webui\models\Unet-onnx\v1-5-pruned-emaonly_81761151.onnx: F:\stable-diffusion-webui\models\Unet-trt\v1-5-pruned-emaonly_81761151_cc61_sample=1x4x64x64+2x4x64x64+8x4x96x96-timesteps=1+2+8-encoder_hidden_states=1x77x768+2x77x768+8x154x768.trt [libprotobuf WARNING **\externals\protobuf\3.0.0\src\google\protobuf\io\coded_stream.cc:604] Reading dangerously large protocol message. If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons. To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h. [libprotobuf WARNING **\externals\protobuf\3.0.0\src\google\protobuf\io\coded_stream.cc:81] The total number of bytes read was 1721604346 [libprotobuf WARNING **\externals\protobuf\3.0.0\src\google\protobuf\io\coded_stream.cc:604] Reading dangerously large protocol message. If the message turns out to be larger than 2147483647 bytes, parsing will be halted for security reasons. To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h. [libprotobuf WARNING **\externals\protobuf\3.0.0\src\google\protobuf\io\coded_stream.cc:81] The total number of bytes read was 1721604346 [W] 'colored' module is not installed, will not use colors when logging. To enable colors, please install the 'colored' module: python3 -m pip install colored [I] Loading tactic timing cache from F:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\timing_caches\timing_cache_win_cc61.cache WARNING:root:Timing cache file F:\stable-diffusion-webui\extensions\Stable-Diffusion-WebUI-TensorRT\timing_caches\timing_cache_win_cc61.cache not found, falling back to empty timing cache. [I] Building engine with configuration: Flags | [REFIT, TF32] Engine Capability | EngineCapability.DEFAULT Memory Pools | [WORKSPACE: 8117.31 MiB, TACTIC_DRAM: 8117.31 MiB] Tactic Sources | [CUBLAS, CUDNN, EDGE_MASK_CONVOLUTIONS, JIT_CONVOLUTIONS] Profiling Verbosity | ProfilingVerbosity.LAYER_NAMES_ONLY Preview Features | [FASTER_DYNAMIC_SHAPES_0805, DISABLE_EXTERNAL_TACTIC_SOURCES_FOR_CORE_0805] Building engine: 50%|█████████████████████████████████▌ | 3/6 [00:00<00:00, 6.14it/s]

socir9 commented 1 month ago

i had same problem i fix it by deleting set CUDA_LAUNCH_BLOCKING=1 SET TORCH_USE_CUDA_DSA=1 this from webui_ser.bat