issues
search
Lightning-AI
/
litgpt
Pretrain, finetune, deploy 20+ LLMs on your own data. Uses state-of-the-art techniques: flash attention, FSDP, 4-bit, LoRA, and more.
https://lightning.ai
Apache License 2.0
6.95k
stars
733
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Add MicroLlama training support
#1457
keeeeenw
opened
9 hours ago
1
Detect tensor cores
#1456
rasbt
closed
18 hours ago
2
Explain how to list all available models
#1455
rasbt
closed
20 hours ago
0
Check checkpoint_dir and add `checkpoints` to path
#1454
rasbt
opened
1 day ago
0
Update error message
#1453
rasbt
closed
1 day ago
0
The difference between FSDPStrategy and DeepSpeedStrategy during pre-training
#1452
wen020
opened
3 days ago
0
Transition checkpoint_dir to root_dir (part of the New CLI transition)
#1451
rasbt
closed
2 days ago
0
Using custom data for `Continue pretraining an LLM`
#1450
SimiPixel
opened
3 days ago
4
A more centered look
#1449
rasbt
closed
4 days ago
0
Mixtral 8x22B support
#1448
SergioG-M
opened
6 days ago
2
Training lasts just 150 seconds for TinyLlama OpenWebtext dataset
#1447
srivassid
opened
6 days ago
0
Specify cache for huggingface openwebtext download
#1446
srivassid
closed
6 days ago
1
How to set max_iters
#1445
srivassid
opened
1 week ago
5
mistralai/Mistral-7B-v0.3 support
#1444
karkeranikitha
opened
1 week ago
2
validation output during finetuning
#1443
richardzhuang0412
closed
4 days ago
2
Update README.md for one last time
#1442
rasbt
closed
1 week ago
0
Upgrade LitData
#1441
rasbt
opened
1 week ago
0
Pin litdata
#1440
rasbt
closed
1 week ago
0
Fix README.md alignment
#1439
rasbt
closed
1 week ago
0
Align readme
#1438
rasbt
closed
1 week ago
0
New CLI
#1437
rasbt
closed
1 day ago
4
Some confusion about weight conversion, as I need to use other engineering to evaluate my LLM
#1436
fireyanci
closed
1 week ago
2
Fix learning rate calculation in pretrain
#1435
rasbt
closed
1 week ago
0
Fix optimizer init with fused=True
#1434
carmocca
closed
1 week ago
0
Create new CI API key
#1433
carmocca
opened
1 week ago
1
pretrain custom dataset gpu memory oom
#1432
wen020
opened
1 week ago
0
Mistral v0.3
#1431
rasbt
opened
1 week ago
0
performing continuous pretraining and then finetuning causes error
#1430
richardzhuang0412
opened
1 week ago
1
Simplify code
#1429
rasbt
closed
1 week ago
1
Is there any best practice for using litdata to load custom data for pretraining?
#1428
wen020
opened
1 week ago
1
how to pretrain llama2 in custom data?
#1427
wen020
closed
1 week ago
1
Streaming for serving with chat's generate function
#1426
rasbt
opened
1 week ago
8
Fix bug where LitData doesn't use seed
#1425
bradfordlynch
closed
1 week ago
0
Adds streaming option to generate
#1424
rasbt
closed
1 week ago
12
Continually pretrained Llama2-7B-hf model inference is not working on 16GB GPU machine
#1423
karkeranikitha
opened
2 weeks ago
5
Resolve output characters garbled
#1422
fireyanci
opened
2 weeks ago
4
WIP: TensorParallel with new strategy
#1421
awaelchli
opened
2 weeks ago
0
Stream option
#1420
rasbt
opened
2 weeks ago
3
Python API
#1419
rasbt
opened
2 weeks ago
0
how to pretrain llama2?
#1418
wen020
closed
1 week ago
4
Lora recipes use lots of memory because of not wrapping parameters with gradient in separate FSDP unit
#1417
RuABraun
closed
2 weeks ago
2
prompt_style
#1416
fireyanci
closed
1 week ago
4
Thunder FSDP as a regular transform
#1415
carmocca
closed
1 week ago
0
Support `no_sync` with Thunder FSDP
#1414
carmocca
closed
2 weeks ago
0
Continue pre-training got RuntimeError: Failed processing /tmp/data
#1413
BestJiayi
opened
2 weeks ago
4
'Phi-3-mini-4k-instruct' is not a supported config name
#1412
georgehu0815
opened
2 weeks ago
1
support for qwen2 and baichuan
#1411
bestpredicts
opened
3 weeks ago
0
Minimal Python example
#1410
rasbt
closed
2 weeks ago
0
OptimizerArgs
#1409
rasbt
closed
1 week ago
10
Redundancy?
#1408
rasbt
closed
3 weeks ago
2
Next