pytorch torchtitan issues

pytorch / torchtitan

A native PyTorch Library for large model training

BSD 3-Clause "New" or "Revised" License

1.26k stars 115 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Modifying memory estimation options and minor changes

#435 sanketpurandare opened 11 hours ago
0
Question about custom cuda operators for tensor parallelism

#434 vermouth1992 opened 21 hours ago
2
Enable CP

#433 fegin opened 1 day ago
0
Add support of DDP and experimental CompiledAutograd

#432 fegin opened 1 day ago
0
Question about Pipeline parallelism

#431 vermouth1992 opened 1 day ago
3
by default disable heavy memory profiling

#430 tianyu-l closed 2 days ago
3
Add the option to turn on async-TP

#429 yifuwang closed 2 days ago
6
Add the option to turn on async-TP

#428 yifuwang closed 3 days ago
0
Add load from HF ckpts to FSDP model fails.

#427 MinghaoYan opened 3 days ago
0
Adding integration test for FSDP Memory Tracking and Estimation

#426 sanketpurandare closed 3 days ago
0
Adding FSDP Memory Tracking and Estimation

#425 sanketpurandare closed 3 days ago
1
Synced estimate.py with train.py

#424 sanketpurandare closed 3 days ago
1
Setting device based on local rank is not robust

#423 awgu closed 3 days ago
1
improve memory profiler to not to profile every iteration

#422 tianyu-l opened 4 days ago
0
LoRA fine-tuning weights explosion in FSDP training

#421 MinghaoYan opened 4 days ago
10
Llama models with custom configurations and uploading to Hugging Face

#420 bkchang opened 4 days ago
1
Set `record_shapes=True` for profiler

#419 awgu closed 4 days ago
0
Improved `repeat_kv` eager perf

#418 awgu closed 3 days ago
1
updates here

#417 H-Huang opened 4 days ago
0
WIP change to run a zero-bubble like schedule

#416 wconstab opened 1 week ago
0
NotImplementedError: aten::nonzero: attempted to run this operator with Meta tensors at loss.backward()

#415 MinghaoYan closed 1 week ago
2
whole_model for fp8

#414 weifengpy closed 1 week ago
0
[DO NOT REVIEW] fsdp fp8-all-gather

#413 weifengpy opened 1 week ago
0
ImportError in LLaMA Training Script

#412 viai957 opened 1 week ago
3
Skip data loading for middle PP ranks

#411 wconstab closed 1 week ago
1
Adding FSDP Memory Tracking and Estimation

#410 sanketpurandare closed 3 days ago
0
DataLoader state is empty for different ranks ?

#409 ahatamiz opened 1 week ago
1
The PyTorch version is incorrect.

#408 Doraemonzzz closed 1 week ago
7
Some testing from me

#407 ad8e opened 1 week ago
3
Prepare train.py for model chunks for pipelining

#406 wconstab closed 1 week ago
3
Will future support include expert parallel and sequence parallel (such as ring attention)?

#405 Doraemonzzz closed 1 week ago
1
enable TritonFusedRMSNorm with local_map annotation

#404 XilunWu closed 2 weeks ago
1
Change debugmodel to have 8 layers

#403 wconstab closed 1 week ago
0
Break down parallelize_llama for inference cases

#402 kwen2501 closed 2 weeks ago
0
SAC API follow ups to restore old behavior

#401 wanchaol closed 2 weeks ago
0
switch to using create_selective_checkpoint_contexts

#400 XilunWu closed 2 weeks ago
1
How to use nsys?

#399 vedantroy opened 2 weeks ago
1
Cosmetic changes to train.py

#398 kwen2501 closed 2 weeks ago
0
Fix SAC BC breaking and renaming to ac_freq

#397 wanchaol closed 2 weeks ago
0
Update unit_test_cpu.yaml with cpu nightly

#396 wanchaol closed 2 weeks ago
0
dump memory snapshot to analyze OOMs

#395 weifengpy closed 1 week ago
2
benchmark perf numbers on H100 GPUs and update performance.md

#394 tianyu-l opened 2 weeks ago
0
enable TP fp8 allgather with PrepareFloat8ModuleInput

#393 wanchaol closed 2 weeks ago
0
update all toml files to use experimental section

#392 wanchaol closed 2 weeks ago
0
del logits=(bs, seq_len, vocab_size) to save 3.9G memory

#391 weifengpy closed 2 weeks ago
1
add the 8-gpu test badge and use correct links for the integration test badges

#390 tianyu-l closed 2 weeks ago
0
fix missing tb logs

#389 tianyu-l closed 2 weeks ago
0
BC fix for ManualPipelineStage import

#388 wanchaol closed 2 weeks ago
0
DeviceMesh BC fix

#387 wanchaol closed 2 weeks ago
1
Abstract out out optimizer params and update foreach calling convention

#386 drisspg closed 3 weeks ago
0