-
Thanks for the project, I have managed to run the project on CPU with decent speed (**6.2 - 6.8 tokens per second**), however, the
the model only generates a small piece of content, and the response…
-
@jakevdp I am trying to use float8 datatypes as part of training quantized spiking neural networks but cannot initialise weights for a network because jax.random.\_uniform does not support jnp.float8_…
-
EfficientDet: Scalable and Efficient Object Detection
* paper: https://arxiv.org/abs/1911.09070v1
> First, we propose a weighted bi-directional feature pyramid network (BiFPN), which allows ea…
-
Example:
```
mistralai/Mistral-7B-v0.1 --quantize bitsandbytes-nf4
```
Request:
```
curl 127.0.0.1:8080/generate \
-X POST \
-d '{"inputs": " You are a helpful assistant What is…
-
We track operations that we need add to TensorFlow Lite here:
- [ ] Conv3d (https://github.com/tensorflow/tensorflow/issues/19658)
- [ ] Quantized TransposeConv (https://github.com/tensorflow/tens…
-
Edit: It looked like someone had `--changeDefaultModel` issue 5 days ago and it was said to be fixed but GPT 4 stays as my default even when it says it's changed:
```
PS C:\Users\slyre> fabric --ch…
-
### Search before asking
- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and fou…
-
## 🚀 Feature
---
In this doc we are requesting comments for the implementation of the sparsification flow as part of architecture optimization namespace (`torch.ao`)
with: @raghuramank100 @dskh…
z-a-f updated
11 months ago
-
There do not appear to be any dequantization functions? There is `meshopt_quantizeHalf` for example, but there is no `meshopt_dequantizeHalf` that turns the quantized value back into a float value.
…
-
Hello, Thank you for providing your code but If I would like to do only post training static quantization then should I use the same code? I mean you're doing the quantized aware training. Please le m…