search

uiuc-focal-lab / syncode

Efficient and general syntactical decoding for Large Language Models
MIT License
200 stars 16 forks source link

Option to add huggingface token for gated models? #79

Closed border-b closed 6 months ago

border-b commented 6 months ago

Currently we can not use models with gated repos from huggingface since they need authentication. Is there a way to pass the huggingface token to access the gated repos from huggingface?

border-b commented 6 months ago

Noticed the token is actually retrieved with os.environ['HF_ACCESS_TOKEN']. So I guess that should work. But I guess it would still be nice to have an option to add the token as an optional argument. Closing the issue.

shubhamugare commented 6 months ago

Hi @border-b,

Yes, the environment variable HF_ACCESS_TOKEN can be used to get under the gated model.

You can also consider importing SynCode logit processor and then can use all HF transformers API as in this notebook. We haven't added README for that but the notebook example should be clear enough.