-
Hi! Is adding FP8 transformer engine (H100) speedup to inference planned?
If not, could you please give me an outline of what needs to be done in order for me to work on that?
Thank you!
-
Hello team,
we have been debugging large scale training instabilities with FP8 and noticed that these started when updating from transfomer-engine v1.2.1 to v1.7. Taking a closer look at the traini…
-
```
Some parameters are on the meta device device because they were offloaded to the cpu.
Quantizing weights: 0%| | 0/1771 [00:00
-
The image generated in f8 mode will report an error
Error occurred when executing T5TextEncode #ELLA:
"index_select_cuda" not implemented for 'Float8_e5m2'
-
如题,已安装了ComfyUI_bitsandbytes_NF4插件。
如果是加载flux1-schnell_fp8_unet_vae_clip模型会出现下面错误
![image](https://github.com/user-attachments/assets/18127d10-29a2-44fc-a62c-0a29bd1fa0a6)
![image](https://github.co…
-
Prompt outputs failed validation
CheckpointLoaderSimple:
- Value not in list: ckpt_name: 'flux1-schnell-fp8.safetensors' not in []
Prompt outputs failed validation
CheckpointLoaderSimple:
- R…
-
### 🚀 The feature, motivation and pitch
Doing matmuls in FP8 is essential for getting the best performance out of the newest hardware (e.g. H100s and beyond). Currently, the only option for doing m…
-
FP8 Linear does not work for me:
> - torch == 2.4.0 + cu121
> - torchao == 0.4.0
> - cuda_arch == 8.9 (nvidia L40)
```python
import torch
import torch.nn as nn
from torchao.float8 import conv…
-
### Your current environment
```text
Collecting environment information...
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
…
-
This is an umbrella issue for allowing fp8 type(s) in shark, spanning all the required layers of the stack: Turbine, IREE, MLIR, LLVM, including backends of interest like ROCm.
Some initial researc…
kuhar updated
7 months ago