search

huggingface / local-gemma

Gemma 2 optimized for your local machine.
Apache License 2.0
238 stars 15 forks source link

authentication in LocalGemma2ForCausalLM with huggingface access token doesn't work #21

Open dsingal0 opened 3 days ago

dsingal0 commented 3 days ago

Authentication in code with token=hf_token doesn't work unless you use subprocess.run("local-gemma", "--token", hf_token, "What is the capital of France")

model = LocalGemma2ForCausalLM.from_pretrained("google/gemma-2-9b", token=hf_token, preset="memory")

sanchit-gandhi commented 3 days ago

Hey @dsingal0! Unfortunately I'm not able to reproduce the error, e.g. using the following example bellow correctly uses the token I specify:

local-gemma --token "hf_..."

The PR #22 should make logging in using your HF more straightforward!

dsingal0 commented 3 days ago

I don't believe the cli has auth issues, but it didn't work in regular python using just LocalGemma2ForCausalLM.

model = LocalGemma2ForCausalLM.from_pretrained("google/gemma-2-9b", token=hf_token, preset="memory")

sanchit-gandhi commented 1 day ago

I see, thanks for the clarification! Could you past the full traceback when you instantiate the LocalGemma2ForCausalLM model please? And also the one you get when you do:

from transformers import Gemma2ForCausalLM

model = LocalGemma2ForCausalLM.from_pretrained("google/gemma-2-9b", token=hf_token)

If possible, you can also call the following from the terminal:

huggingface-cli login

And then enter you auth token to cache. You then won't have to pass the token argument to from_pretrained.

dsingal0 commented 1 day ago

I can't do huggingface-cli login since I'm using baseten.co to deploy gemma2, and unless I store my HF API token as an envvar as opposed to a docker secret it isn't accessible before I'm in python code, so I'd have to do subprocess.run.

ilarramendi commented 9 hours ago

huggingface-cli login worked for me, the token parameter seems to be not working