-
# LoRA: Low-Rank Adaptation of Large Language Models
基于large pre-trained model,把基于某个任务的微调存储在低秩矩阵对中,low intrinsic dimension $r=4$ 就够。
Pro:
- 并行化不影响速度、任务特化的信息相对很少。
- 该方法对超参数极其不敏感。
另外:
- 对于模型…
-
### Issue Type
Build/Install
### Modules Involved
MPC protocol
### Have you reproduced the bug with SPU HEAD?
Yes
### Have you searched existing issues?
Yes
### SPU Version
spu 0.9.0.dev20240…
-
Hello!! I'm excited when I meet the nengo project!! I want to simulate my neuron model in nengo_loihi or nengo_FPGA. However, my neuron model can fire negative spike. I know the nengo support negative…
-
### 🐛 Describe the bug
Hello,
I am running llama3-70b and mixtral with VLLM on a bunch of different kinds of machines. I encountered wildly different quality performance on A10 GPUs vs A100/H…
-
### Search before asking
- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and fou…
-
Papers:
- Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorization https://arxiv.org/abs/1902.01917
- Up or Down? Adaptive Rounding for Post-Training …
-
Hi,
I trained YOLOv8 model and exported the model to ONNX format by the quantization_recipe below, I set weight_bits=8 and activation_bits=8 to ensure the full-flow inference of quantized model is …
-
The [toturial](https://pytorchvideo.org/docs/tutorial_accelerator_build_your_model) shows how to build an efficient network with modules provided by "pytorchvideo.layers.accelerator" and how to conver…
-
### Search before asking
- [X] I have searched the YOLOv8 [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and fou…
-
I am trying to save a quantized ternary model to a `.tflite` file, but larq doesn't seem to save the weights using datatypes with a reduced precision and thus compress the file size.
However, after c…