issues
search
dvlab-research
/
LongLoRA
Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)
http://arxiv.org/abs/2309.12307
Apache License 2.0
2.59k
stars
267
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
LongBench evaluation
#194
Clement25
opened
1 week ago
0
是否支持如GPT2这类的supervised fine-tune?
#193
CharRic
opened
2 weeks ago
0
How LongAlpaca Data was constructed?
#192
S1s-Z
opened
1 month ago
0
这套代码是否支持qwen/baichuan微调一个中文的长文本模型,代码需要做哪些修改?
#191
jy-101361-1810897
opened
1 month ago
0
norm层不是没有参数矩阵吗
#190
changanxunyi
opened
3 months ago
0
Update README.md
#189
Dominic789654
opened
3 months ago
0
I am unable to reproduce the results from the paper for llama-7B-32k-longlora ppl.
#188
masteryqq
opened
3 months ago
1
模型完全没法正常输出
#187
Tangent-90C
closed
3 months ago
1
embedding 为什么要resize成32001?
#186
momandai
opened
3 months ago
0
Something wrong with the torch version
#185
dian1414
opened
3 months ago
0
What's the trainset is used to obtain “Model with contextg extension via improved LoRA fine-tuning” (LoRA+)?
#184
ZackZikaiXiao
opened
4 months ago
0
How did make questions and answers for long context(LongAlpaca)?
#183
ddoyles
opened
6 months ago
0
When I set `per_device_train_batch_size=2`, the S2-Attn would not shift as expected
#182
linhaojia13
opened
6 months ago
2
HF models missing rope scaling in the config
#181
hsiehjackson
opened
6 months ago
0
Machine don't install Flash Attention
#180
huilong-chen
opened
6 months ago
0
global_step文件
#179
xxcoco763
opened
6 months ago
0
Add callback for saving trainable parameters and model config
#178
GirinMan
opened
6 months ago
0
Regarding the results in Table 8 and Table 14
#177
Statisticss
opened
7 months ago
0
About the different datasets and corresponding models
#176
Statisticss
opened
7 months ago
0
The proof-pile/test-sample-ids is not the exact ids for the proof-pile-testsample.bin
#175
pangjh3
closed
7 months ago
0
Memory usage "too small" for 7B Llama-2
#174
Linohong
opened
7 months ago
0
training a LLM w/ shifted sparse attention from the scratch?
#173
we1k
opened
7 months ago
0
merge_lora_weights_and_save_hf_model.py Error while deserializing header: HeaderTooLarge
#172
Spongeorge
opened
7 months ago
0
Distributed inference issue
#171
yixliu1
opened
7 months ago
0
论文中的evaluate结果,推理时用的attention是shifted sparse attention?还是full attention?
#170
zhangxiann
opened
7 months ago
0
Is it possible to increase the context length of phi-2 using LongLora? If yes, what changes need to be done to support it?
#169
dbanka
opened
7 months ago
1
the value of loss is too unstable when supervised-finetune the 7b-100k-ft model
#168
seanxuu
opened
7 months ago
1
streaming llm problem
#167
seanxuu
opened
7 months ago
0
How can I use the Llama-2-7b-longlora-100k-ft model correctly
#166
seanxuu
opened
7 months ago
0
bug report : RuntimeError: probability tensor contains either inf, nan or element < 0
#165
seanxuu
opened
7 months ago
0
Is LongLoRA can be mixed with YaRN ?
#164
DevNullx64
opened
8 months ago
0
推理时候显存分配
#163
xxcoco763
opened
8 months ago
2
Adapting to new models
#162
epinnock
opened
8 months ago
2
如何在LoRA训练中加入embed和norm层的训练?
#161
Zheng-Jay
opened
8 months ago
0
Lora+deepspeed zero3 无法保存lora权重问题
#160
AresXD
closed
8 months ago
6
What llama attn replacement to use for SFT-based inference?
#159
spring1915
opened
8 months ago
0
在没有报错的情况下,LongAlpaca-7B只对文本的第一段文字进行了响应
#158
waleyW
opened
8 months ago
0
Configs in inference.py necessary for context length expansion in model serving?
#157
spring1915
opened
8 months ago
0
训练的时候使用的什么外推方式
#156
IT-five
opened
8 months ago
0
支持qwen、baichuan等中文模型微调吗
#155
kevinuserdd
opened
8 months ago
0
inference OOM
#154
PharMolix
opened
9 months ago
0
Is LongAlpaca model fine-tuned from llama-2 or the Alpaca model?
#153
Mooler0410
opened
9 months ago
0
Can LongLoRA be used for incremental pre-training?
#152
Zheng-Jay
opened
9 months ago
0
the current text generation call will exceed the model's predefined maximum length (4096)
#151
waleyW
opened
9 months ago
4
微调数据
#150
Go4miii
closed
9 months ago
0
推理 group整除问题
#149
Michelleable
closed
9 months ago
1
LongLoRA + Flash Attention 2 causing illigal memory access
#148
ArturNiederfahrenhorst
opened
9 months ago
7
32k inference result is garbled
#147
zhanglv0209
opened
9 months ago
8
torch.cuda.OutOfMemoryError: CUDA out of memory
#146
zhanglv0209
closed
9 months ago
3
中文领域进展
#145
ccp123456789
closed
9 months ago
1
Next