chengzeyi
commented
7 months ago @rbhuta Seems that you don't have a development version of Python properly installed on your system. You need to install python3.10-dev with your system's package manager.
rbhuta
Hello, I was able to get this repo working with sdxl turbo on my 4090 using a venv, but I tried to dockerize the build and it repeatedly fails on a missing tmp file. I am wonder if you have seen this issue before and if I perhaps am missing some apt or pip installs that are missing in my docker image causing this issue. Appreciate your help @chengzeyi.
Dockerfile: FROM nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ --mount=type=cache,target=/var/lib/apt,sharing=locked \ apt-get -y update \ && apt-get install -y --no-install-recommends python3.10 python-is-python3 git libgl1 libsndfile1 pip ffmpeg google-perftools \ libvulkan1 libnvidia-gl-525-server mesa-vulkan-drivers gcc build-essential \ && apt-get autoremove -y \ && apt-get clean \ && rm -rf /var/lib/apt/lists/*
RUN pip install --upgrade pip setuptools wheel --no-cache-dir
WORKDIR testing COPY requirements2.txt requirements2.txt RUN pip install -r requirements2.txt --no-cache-dir
COPY . .
CMD ["python3", "examples/optimize_stable_diffusion_pipeline.py"]
Error logs:
INFO:root:Tracing forward /usr/local/lib/python3.10/dist-packages/sfast/utils/flat_tensors.py:159: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! obj_type = tensors[start].item() /usr/local/lib/python3.10/dist-packages/sfast/utils/flat_tensors.py:218: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! size = tensors[start].item() /usr/local/lib/python3.10/dist-packages/sfast/utils/flat_tensors.py:228: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! size = tensors[start].item() /usr/local/lib/python3.10/dist-packages/sfast/utils/flat_tensors.py:214: TracerWarning: Converting a tensor to a Python list might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! return bytes(tensors[start].tolist()), start + 1 /usr/local/lib/python3.10/dist-packages/transformers/modeling_attn_mask_utils.py:66: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if input_shape[-1] > 1 or self.sliding_window is not None: /usr/local/lib/python3.10/dist-packages/transformers/modeling_attn_mask_utils.py:137: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if past_key_values_length > 0: /usr/local/lib/python3.10/dist-packages/transformers/models/clip/modeling_clip.py:273: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if attn_weights.size() != (bsz self.num_heads, tgt_len, src_len): /usr/local/lib/python3.10/dist-packages/transformers/models/clip/modeling_clip.py:281: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if causal_attention_mask.size() != (bsz, 1, tgt_len, src_len): /usr/local/lib/python3.10/dist-packages/transformers/models/clip/modeling_clip.py:313: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if attn_output.size() != (bsz self.num_heads, tgt_len, self.head_dim): /usr/local/lib/python3.10/dist-packages/sfast/utils/flat_tensors.py:23: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. return torch.tensor([num], dtype=torch.int64) /usr/local/lib/python3.10/dist-packages/sfast/utils/flat_tensors.py:253: TracerWarning: torch.Tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. return super().new(cls, x, args, kwargs) /usr/local/lib/python3.10/dist-packages/sfast/utils/flat_tensors.py:123: TracerWarning: torch.as_tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. return (torch.as_tensor(tuple(obj), dtype=torch.uint8), ) 0%| | 0/30 [00:00<?, ?it/s]INFO:root:Dynamically graphing forward /usr/local/lib/python3.10/dist-packages/torch/cuda/graphs.py:88: UserWarning: The CUDA Graph is empty. This usually means that the graph was attempted to be captured on wrong device or stream. (Triggered internally at ../aten/src/ATen/cuda/CUDAGraph.cpp:192.) super().capture_end() INFO:root:Tracing forward /usr/local/lib/python3.10/dist-packages/sfast/utils/flat_tensors.py:197: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! return bool(tensors[start].item()), start + 1 /usr/local/lib/python3.10/dist-packages/diffusers/models/unet_2d_condition.py:878: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if dim % default_overall_up_factor != 0: /usr/local/lib/python3.10/dist-packages/diffusers/models/resnet.py:265: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert hidden_states.shape[1] == self.channels /usr/local/lib/python3.10/dist-packages/diffusers/models/resnet.py:271: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert hidden_states.shape[1] == self.channels /usr/local/lib/python3.10/dist-packages/diffusers/models/resnet.py:173: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert hidden_states.shape[1] == self.channels /usr/local/lib/python3.10/dist-packages/diffusers/models/resnet.py:186: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if hidden_states.shape[0] >= 64: /tmp/tmpuumtrxyy/main.c:4:10: fatal error: Python.h: No such file or directory 4 | #include
| ^
output_image = model( kwarg_inputs).images[0]
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(args, kwargs)
File "/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py", line 918, in call
noise_pred = self.unet(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, *kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(args, kwargs)
File "/usr/local/lib/python3.10/dist-packages/sfast/cuda/graphs.py", line 29, in dynamic_graphed_callable
cached_callable = simple_make_graphed_callable(
File "/usr/local/lib/python3.10/dist-packages/sfast/cuda/graphs.py", line 46, in simple_make_graphed_callable
return make_graphed_callable(callable,
File "/usr/local/lib/python3.10/dist-packages/sfast/cuda/graphs.py", line 75, in make_graphed_callable
callable(tree_copy(example_inputs),
File "/usr/local/lib/python3.10/dist-packages/sfast/jit/trace_helper.py", line 55, in wrapper
return traced_module(args, kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, *kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(args, kwargs)
File "/usr/local/lib/python3.10/dist-packages/sfast/jit/trace_helper.py", line 112, in forward
outputs = self.module(self.convert_inputs(args, kwargs))
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, *kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(args, kwargs)
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
~~~~~ compilation terminated. 0%| | 0/30 [00:01<?, ?it/s] Traceback (most recent call last): File "/testing/examples/optimize_stable_diffusion_pipeline.py", line 81, ingraph(%input, %num_groups, %weight, %bias, %eps, %cudnn_enabled): %y : Tensor = sfast_triton::group_norm_silu(%input, %num_groups, %weight, %bias, %eps)