-
Hi Team,
Looks like Gemma 2 is not supported by candle yet
-
What would be required to support Gemma 2?
I'd be happy to chip in and help with the code, I just need to have a bit of insight into what would need to be changed?
-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
- `llamafactory` version: 0.9.1.dev0
- Platform: Linux-3.10.0-957.el7.x86_64-x86_64-with-glibc2.35
- Py…
-
### Your current environment
```text
Versions of relevant libraries:
[pip3] flashinfer==0.0.9+cu121torch2.3
[pip3] numpy==1.26.4
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] sentence-transformers==3.0…
-
Hi @danielhanchen ,
I am unable to use "unsloth/gemma-2b-bnb-4bit" via vLLM. I am getting below error while loading the model on Nvidia_T4 or NVIDIA_V100 GPU .
`engine_args = EngineArgs(model="u…
-
### Contact Details
jeezricklovelife@gmail.com
### What happened?
When I use llamafile with python api. But for 2 models I use, they all retain the end token in response string, that I need to manu…
-
I tried to run this with Gemma 2 27b it and found that it doesn't quite work. I verified that everything works with qwen/qwen-1_8b-chat.
I get this error message:
```Assertion error: All scores…
-
from_finetuned() doesnt load any adapter bin after finetuning gemma
-
Currently our configs have:
```
output_dir: /tmp/alpaca-gemma-finetune
checkpointer:
checkpoint_dir: /tmp/gemma-2b/
output_dir: /tmp/gemma-2b
metric_logger:
log_dir: ${output_dir}
…
-
This could be question rather than a feature request.
flashinfer is not supported for AMD GPUs and it's not currently planned until a [later version](https://github.com/flashinfer-ai/flashinfer/iss…