-
-
Hi @tridao ,
I'd like to use the value of [softmax_lse](https://github.com/Dao-AILab/flash-attention/blob/main/flash_attn/flash_attn_interface.py#L1117) in my model and back-propagate gradient thro…
-
### 🐛 Describe the bug
Setting the wrong value `True` to `dim` of [Softmax()](https://pytorch.org/docs/stable/generated/torch.nn.Softmax.html) gets the error message as shown below:
```python
imp…
-
flashdecoding++ paper: https://arxiv.org/abs/2311.01282
- Q3 Collaboration Plan of Infra and IaaS Labs: https://bytedance.us.larkoffice.com/docx/HKXfdRh1noMrbAxcgL2ureGasdQ
- FlashDecoding++ Su…
-
# Bug Report
### Describe the bug
I use quantize_static and convert_float_to_float16 functions in onnx to convert fp32 model to fp16 + int8 model. The fp16 model can inference through onnxruntime …
Ai-ZL updated
2 weeks ago
-
I have been trying to install all the dependencies. Only diff-gaussian-rasterization-softmax submodule is problematic. I have cuda 11.8 installed in my 22.04 ubuntu system, and when i do `pip install …
-
-
### 🚀 The feature, motivation and pitch
sparse.softmax apparently only supports sparse COO format, while sparse CSR is presented as more efficient and default format for other operations.
For examp…
-
**What is your question?**
When I run `examples/35_gemm_softmax`, I use `nvcc --expt-relaxed-constexpr -I /mnt5/xuantai.hxd/cutlass/include -I /mnt5/xuantai.hxd/run_cutlass/cutlass_gemm_softmax -…
-
Why multiply 5?