issues
search
BlackSamorez
/
tensor_parallel
Automatically split your PyTorch models on multiple GPUs for training & inference
MIT License
629
stars
39
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Compatibility with `transformers > 4.36`: error: `AttributeError: 'tuple' object has no attribute 'to_legacy_cache'`
#137
Dr-Left
opened
3 months ago
2
Customized generate func support?
#136
MonolithFoundation
opened
6 months ago
0
Add mixtral support
#134
ReinForce-II
opened
10 months ago
0
tensor_parallel int4 LLM is not working since release v2.0.0
#133
ReinForce-II
opened
10 months ago
0
Now, does tensor_parallel no longer support the huggingface trainer?
#132
HanGyeol-Yoo
opened
11 months ago
0
Can I use tensor_parallel to inference for a GPTQ quantized model?
#131
minlik
opened
1 year ago
0
No implement of generate() when using models from hugging face.
#130
342215448
opened
1 year ago
0
TensorParallel object has no attribute save_pretrained
#129
toufunao
opened
1 year ago
0
No output when using tensor_parallel
#128
yyya9
opened
1 year ago
1
How to use the model in a scenario where it is stored in the Safetenors format?
#127
yxk9810
closed
1 year ago
0
Out of GPU memory for two A10 GPUs
#126
JunyiYe
closed
1 year ago
1
AttributeError: object has no attribute 'devices'
#125
QiueY514
opened
1 year ago
0
ValueError: Model parameters were moved to incorrect devices, did call on model.cuda() or model.to(device)? If so, please avoid doing that
#124
Khyat
opened
1 year ago
0
fix recursion error when setting tp_wrapped_module #122
#123
Ar-Kareem
opened
1 year ago
4
Max Recursion Error when using with lora
#122
Ar-Kareem
opened
1 year ago
2
RuntimeError: NCCL Error 3: internal error
#121
smallmocha
opened
1 year ago
1
Segmentation fault (core dumped)
#120
jameswu2014
opened
1 year ago
0
Support of 8-bit and 4-bit quantization
#119
ludwigflo
closed
1 year ago
1
Would it suitable for the multi-GPU parallel inference for llama2?
#118
aclie
opened
1 year ago
0
2x slowdown using TP
#117
jph00
opened
1 year ago
0
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
#116
SparkJiao
opened
1 year ago
0
distributed TP model forward output's requires_grad is False
#115
lxuechen
opened
1 year ago
5
tensor_parallel method distributed=True
#114
Johnno1011
opened
1 year ago
2
Forwarding _prepare_model_inputs
#113
BlackSamorez
closed
1 year ago
0
model.generate() with inputs_embeds
#112
ZhaoxuanWu
closed
1 year ago
3
Fix false positive in tests for finding predefined TP config
#111
tonywang16
closed
1 year ago
2
find_predefined_tensor_parallel_config try-except fix
#110
BlackSamorez
closed
1 year ago
0
Testing interfaces (soon to be refactored)
#109
BlackSamorez
closed
1 year ago
0
Fix get_llama_config adding model attribute error.
#108
tonywang16
closed
1 year ago
3
Error loading LLAMA model config
#107
tonywang16
closed
1 year ago
0
[WIP] ZeRO-3 refactoring (sharding)
#106
BlackSamorez
closed
1 year ago
1
When I try to do the tensor_parallel on NLLB from meta, there is an error
#105
342215448
opened
1 year ago
0
When I try to do the tensor_parallel on NLLB from meta, there is an error:
#104
342215448
closed
1 year ago
1
Gpt2 fix
#103
BlackSamorez
closed
1 year ago
1
Version bump
#102
BlackSamorez
closed
1 year ago
0
LLaMA-2
#101
BlackSamorez
closed
1 year ago
1
explicitly choose whether or not to use torch.distribute
#100
tomoki0924
closed
1 year ago
2
GPT-2 broken starting in v1.2.5
#99
eric-mitchell
closed
1 year ago
1
Issues if GPU > 2
#98
Tom-Ryder
closed
1 year ago
6
Cloud Tensor_parallel add multiple accelerator inference support with torch.distributed?
#97
hijeffwu
closed
1 year ago
4
Example Question (got error) : Try new 40B LLMs demo in Kaggle
#96
YooSungHyun
closed
1 year ago
2
why raised cuda error?
#95
YooSungHyun
closed
1 year ago
18
Possibility to run on different GPUs
#94
Ch4mpa9ne
closed
1 year ago
2
readme fixes
#93
BlackSamorez
closed
1 year ago
0
Falcon lm_head split hotfix
#92
BlackSamorez
closed
1 year ago
0
Falcon predefined config
#91
BlackSamorez
closed
1 year ago
0
Fixed dispatch of tp.Sharded models
#90
BlackSamorez
closed
1 year ago
0
TypeError when multi-thread inference using tensor_parallel
#89
liulhdarks
closed
1 year ago
1
Question on custom models
#88
vince62s
opened
1 year ago
23
Removing PEFT from dependencies. Replacing with runtime checks
#87
BlackSamorez
closed
1 year ago
0
Next