lyogavin airllm issues - Githubissues

lyogavin / airllm

AirLLM 70B inference with single 4GB GPU

Apache License 2.0

5.09k stars 408 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Add support for Mistral model inference

#146 kunling-cxk opened 4 months ago
0
ImportError: cannot import name 'AutoModel' from partially initialized module 'airllm' (most likely due to a circular import)

#145 leobilocastro closed 4 months ago
0
Linear(in_features=28672, out_features=8192, bias=False) does not have a parameter or a buffer named qweight.

#144 luzacao opened 5 months ago
0
WeChat QR Code out of date

#143 zixianwang2022 opened 5 months ago
0
air_llm: README fix MacOS typo

#142 hiemal closed 5 months ago
0
safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer

#137 chuangzhidan opened 6 months ago
0
Insuficient disk space

#136 ulisesbussi opened 6 months ago
3
CPU ram offload

#135 NicolasMejiaPetit opened 6 months ago
0
error in apple mac m3

#134 mustangs0786 opened 6 months ago
5
Does airllm support quantized gguf/gptq/awq models ?

#133 robik72 opened 6 months ago
0
COMPILED_WITH_CUDA error requires libcuda.so

#132 nickums opened 6 months ago
0
Error with Llama3: ValueError: Trying to set a tensor of shape torch.Size([1024, 8192]) in "weight" (which has shape torch.Size([8192, 8192])), this look incorrect.

#131 Cangshanqingshi closed 6 months ago
0
跑不通chatglm3，请大佬指教。

#130 ZiQiangXie opened 6 months ago
2
segmentation fault python3 airllm2.py

#129 taozhiyuai opened 6 months ago
3
to run llama3-70b,but fail to import. why?

#128 taozhiyuai closed 6 months ago
0
Any CoreML implementation plans?

#127 Proryanator opened 6 months ago
0
Mac 'str' object has no attribute 'sequences

#126 gr3enarr0w opened 6 months ago
0
"src" directory name is conflicted

#125 Rambo55555 opened 6 months ago
0
how to delete the original download model after it has been downloaded

#124 ruiguo-bio opened 6 months ago
1
Running on Mac get traceback error

#123 gr3enarr0w closed 6 months ago
3
通过Ollama下载了的模型，如何在airllm中直接使用呢

#122 w1005444804 opened 6 months ago
2
请求支持llama3

#121 CrazyBoyM closed 6 months ago
2
The following error is encountered when running the sample code

#120 Nuclear6 opened 7 months ago
0
compression parameter on mac.dosent work.

#119 dnvs opened 7 months ago
0
Support for OPT Architecture

#118 varunlmxd opened 7 months ago
0
Is it possible to use AirLLM with a quantized input model?

#117 Verdagon opened 7 months ago
3
mac m2 run air llm garage-bAInd/Platypus2-7B get error Input must be a file-like object opened in binary mode, or string

#116 wuxiongwei opened 8 months ago
6
似乎只能产生很少的字符

#115 andeyeluguo closed 8 months ago
2
Add UI like AUTOMATIC1111 for stable-diffusion-webui

#114 janmartin opened 8 months ago
0
Which 70B model does macOS support?

#112 ruifengma opened 8 months ago
0
Generation takes forever

#111 Kira-Pgr closed 2 months ago
4
Optimize for consumer GPU, eg 11GB or 16GB

#109 profintegra opened 9 months ago
0
AirLLM: Support for DirectML

#108 vegax87 opened 9 months ago
1
attn impl to sdpa...

#107 saa1028 opened 9 months ago
4
AMD gpu support

#106 hanq-moreh opened 9 months ago
2
For me this model is extremely underperforming

#105 SadafShafi opened 9 months ago
1
Macbook "Torch not compiled with CUDA enabled" Error

#104 LanLanBoom closed 9 months ago
2
用airllm运行Yi-34B-chat模型，分层之后报这个错误

#103 peiyanyang opened 9 months ago
1
Will the airllm framework be adapted for the streaming output functionality of different models in the future?

#102 wangqn1 opened 9 months ago
0
ValueError: LlamaForCausalLM does not support an attention implementation through torch.nn.functional.scaled_dot_product_attention yet.

#101 sleeper1023 opened 9 months ago
1
AirLLMLlamaMlx fails to load model with mlx==0.0.7

#100 jakule opened 10 months ago
0
关于对话模型是否能使用airllm

#99 wzz981 opened 10 months ago
1
how to infer on multiple gpus?

#98 yuxx0218 closed 10 months ago
1
Fix TYPO

#97 Naozumi520 closed 10 months ago
0
Finetune 70B on 24GB 4090?

#96 Naozumi520 opened 10 months ago
1
microsoft-phi2:max() arg is an empty sequence

#95 zazaji opened 10 months ago
1
ImportError: cannot import name AutoMode

#94 zazaji closed 10 months ago
1
safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge

#93 fudp opened 10 months ago
1
ValueError: max() arg is an empty sequence(Apple M2 Max, macOS 14.2.1)

#91 tvsj opened 10 months ago
6
Discord Invite Expired in the readme

#90 birdup000 opened 10 months ago
1

Previous Next