issues
search
pytorch-labs
/
gpt-fast
Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.
BSD 3-Clause "New" or "Revised" License
5.35k
stars
484
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
add int4 cpu support
#84
mingfeima
closed
5 months ago
7
[not for land] Temp fix to make GPTQ work
#83
HDCharles
opened
5 months ago
0
Fixes for eval and GPTQ after move to gpt-fast
#82
HDCharles
closed
5 months ago
0
torch.compile leads to OOM with different prompts.
#81
samuelstevens
opened
5 months ago
0
generate.py: do not use args in function main
#80
guoyejun
opened
5 months ago
0
intel gpu : enable intel gpu
#79
xiaowangintel
opened
5 months ago
2
Code is extremely slow!
#78
yafehlis
opened
5 months ago
1
Inference on a dataset instead of an individual prompt
#77
yafehlis
closed
5 months ago
0
added TinyLlama flexibility
#76
yafehlis
closed
5 months ago
2
Error when running convert_hf_checkpoint.py for TinyLlama-1.1B-intermediate-step-480k-1T
#75
yafehlis
closed
5 months ago
0
TypeError: __init__() got an unexpected keyword argument 'mmap'
#74
yafehlis
closed
6 months ago
1
DCFormer training and inference
#73
mqyqlx
closed
6 months ago
1
Does `gpt-fast` work on V100 GPUs?
#72
RomanKoshkin
opened
6 months ago
2
Support Mixtral-8x7B
#71
yanboliang
closed
4 months ago
7
Phi 2
#70
vinhtran2611
closed
6 months ago
1
Device-side assertions’ error when speculative decoding with different length of prompts.
#69
ZipECHO
opened
6 months ago
0
'Triton Error [CUDA]: device kernel image is invalid' while compiling
#68
Armod-I
opened
6 months ago
2
Check PyTorch version
#67
yifuwang
opened
6 months ago
0
Set cuda device before init_process_group
#66
yifuwang
closed
6 months ago
0
repeat sentence and non-complete sentence in the end
#65
allen-ash
opened
6 months ago
0
8 (or 2 more) X A100 GPUs Model Output is Garbled and Failure to Terminate the Program Properly (One GPU is Correct)
#64
qianghuangwhu
opened
6 months ago
6
added presets for mistral7b
#63
alvion427
opened
6 months ago
5
Expert parallelism / MoE example would be awesome :)
#62
andersonbcdefg
opened
6 months ago
1
blip2 can be supportted??
#61
wangjing60755
opened
6 months ago
0
fix safetensors
#60
152334H
opened
6 months ago
1
Understanding why TorchInductor cannot speed-up huggingface transformer inference
#59
learning-chip
closed
4 months ago
5
What is `torch.ops.aten._convert_weight_to_int4pack` ?
#58
vgoklani
closed
4 months ago
5
Add mixtral support
#57
Chillee
opened
6 months ago
1
Set cuda device before init_process_group
#56
yifuwang
closed
6 months ago
3
Tensor parallel hangs on call to model
#55
briandw
closed
6 months ago
6
Bug convert HF model
#54
vinhtran2611
opened
6 months ago
3
Too long input texts cuase device-side assert triggered
#53
li-aolong
opened
6 months ago
1
Allow small modes to work with convert_hf_checkpoint. Added TinyLLama to the model list
#52
briandw
opened
6 months ago
0
AttributeError: torch._inductor.config.fx_graph_cache does not exist
#51
chinmay29
opened
6 months ago
1
slight performance improving(ㄒoㄒ)
#50
480284856
opened
6 months ago
1
RuntimeError: cutlassF: no kernel found to launch!
#49
goodboyyes2009
opened
6 months ago
15
Add a "Community" section in README.
#48
huntzhan
closed
6 months ago
1
torch.compile() with flash decoding ops
#47
rayleizhu
opened
6 months ago
7
pytorch版本问题,运行整套流程torch版本需要特定的版本吗?还是说2.1.0以上就可以
#46
Joker-sad
opened
6 months ago
3
Can we run gpt-fast from Windows Command Prompt or Powershell?
#45
maxloosmu
closed
6 months ago
3
Support ScalingRotaryEmbedding
#44
briandw
opened
6 months ago
0
How to cache the compilation result?
#43
huntzhan
opened
6 months ago
2
Unexpected key(s) in state_dict: "rope.freqs".
#42
Prakash19921206
opened
6 months ago
1
Update README.md
#41
54yyyu
opened
6 months ago
0
Update README.md
#40
54yyyu
closed
6 months ago
1
Is torch.empty_like truely random?
#39
MasterGodzilla
closed
6 months ago
1
Mistral support
#38
Nikita-Sherstnev
closed
4 months ago
2
Support code gen for non-cuda targets with gpt-fast
#37
mikekgfb
closed
6 months ago
6
May I ask which version of PyTorch does this project correspond to?
#36
ye1024
closed
6 months ago
3
What would it take to support other models like deepseek coder?
#35
briandw
closed
6 months ago
3
Previous
Next