-
I'll just put this in an issue to avoid getting things too mixed up. I've added `list` as a valid return type from `ExLlamaV2Filter.next()` in the dev branch, and it does indeed reduce the overhead qu…
-
Hi, I run the training on IWSLT17 En-De translation dataset, but got following 'None' loss without raising any exception. Could you help? Thanks!
2023-02-23 11:14:21 | INFO | train | epoch 001 | lo…
-
Amazing job, I wonder the minimum GPU VRAM for training, image size is 512 and batch size is one or two
-
as the title
-
请教relative_transformer.py中_transpose_shift函数中的几个问题:
(1)它实现了矩阵的什么变换(如移动、旋转等)?
(2)怎么理解它是如何实现这种变换的呢?
(3)倒数第3行中的indice为什么只选取奇数行呢?
(4)_transpose_shift函数与_shift有什么区别,又有什么联系?
十分感谢
```
def _transpo…
-
_From [rnk@google.com](https://code.google.com/u/rnk@google.com/) on November 18, 2011 13:34:41_
This happens during unit_tests.exe shutdown when run without -quiet and -no_results_to_stderr as on th…
-
Trying to finetune unsloth/Mistral-Nemo-Instruct-2407-bnb-4bit. It gives me error with batch size 1 and max seq length of 2048.
I can see the example notebook on colab t4.
Unsloth: Fast Mistral pa…
-
When I'm doing the evaluation, should I use _--load_8bit_? I'm trying to reproduce the results of LLaMa-7B-LoRA
Finetune:
`CUDA_VISIBLE_DEVICES=8 python finetune.py --base_model 'yahma/llama-7b-…
-
### 🐛 Describe the bug
# the bug description
When I use the tensor shape like this (batch_size, seq_len, embedding_size) and i put the batch_size, seq_len in dynamic_axes to generate onnx model, I c…
-
Hi, I would like to ask why the attention mask is not used in the prefill stage.
I want to output the attention scores matrix in prefill stage. Is the code below right?
```
if spec: # s…