issues
search
davmacario
/
MDI-LLM
Implementation of Model-Distributed Inference for Large Language Models, built on top of LitGPT
MIT License
3
stars
2
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Final version - thesis
#39
davmacario
closed
3 months ago
0
Fix model support (Mistral 7B)
#38
davmacario
closed
4 months ago
0
Separate socket handling from GPTServer
#37
davmacario
closed
4 months ago
0
Litgpt refactor
#36
davmacario
closed
4 months ago
0
Support for half-precision in Llama
#35
davmacario
closed
4 months ago
0
Add Llama training script
#34
davmacario
closed
4 months ago
0
Add possibility to offload inactive KV Caches to RAM
#33
davmacario
opened
5 months ago
0
Inspect high VRAM usage in Llama 3
#32
davmacario
closed
4 months ago
3
Add support for Llama architecture
#31
davmacario
closed
5 months ago
0
Add "smart" layer partition
#30
davmacario
opened
5 months ago
0
Switch to MIT license
#29
davmacario
closed
6 months ago
0
Add support for model-parallel training
#28
davmacario
opened
6 months ago
1
Improve readme
#27
davmacario
closed
6 months ago
0
Add output queue
#26
davmacario
closed
6 months ago
0
Add prompt support
#25
davmacario
closed
6 months ago
0
Add support for generating more than `n_nodes` samples
#24
davmacario
closed
6 months ago
0
Solve issue when training on second GPU
#23
davmacario
closed
6 months ago
0
Develop
#22
davmacario
closed
6 months ago
0
Fix issue when training on "second" gpu (cuda:1)
#21
davmacario
closed
6 months ago
1
Fix memory usage when loading model from chunks
#20
davmacario
closed
4 months ago
1
Adopt transmission queue
#19
davmacario
closed
6 months ago
0
[feat]: replace intermediate and finisher nodes with 'secondary' nodes
#18
davmacario
closed
7 months ago
0
Move ln_l to starter node and replace intermediate & finisher with "secondary" node
#17
davmacario
closed
7 months ago
0
Move ln_l to starter node and replace intermediate & finisher with "secondary" node
#16
davmacario
closed
7 months ago
0
Move `ln_f` (final normalization) to starter node
#15
davmacario
closed
7 months ago
0
Minor updates
#14
davmacario
closed
7 months ago
0
Add docs + improve readme
#13
davmacario
closed
7 months ago
0
Add prompt support for MDI
#12
davmacario
closed
6 months ago
0
[feat]: add GPT2 implementation and support for chunks
#11
davmacario
closed
7 months ago
0
Add support for generating > n_nodes samples
#10
davmacario
closed
6 months ago
0
Add possibility to assign different devices to different nodes running on the same host
#9
davmacario
closed
7 months ago
0
Torch >= 2.2.0 bug on MPS
#8
davmacario
opened
7 months ago
0
Add possibility to override torch device from command line
#7
davmacario
closed
6 months ago
0
Adopt Conv1D layers instead of Linear layer
#6
davmacario
closed
4 months ago
1
[bug]: reduce memory usage, the correct way
#5
davmacario
opened
7 months ago
1
[feat]: final stable version of *my* nanoGPT flavor
#4
davmacario
closed
7 months ago
0
[FEAT]: Implement new tokenizer (tested)
#3
davmacario
closed
7 months ago
0
[FEAT]: first working model, tested, using NanoGPT
#2
davmacario
closed
8 months ago
0
[feat]: swap client/server nodes
#1
davmacario
closed
8 months ago
0