search

uiuc-focal-lab / syncode

Efficient and general syntactical decoding for Large Language Models
MIT License
181 stars 10 forks source link

Expose Syncode Logitsprocessor through syncode API #76

Closed shubhamugare closed 4 months ago

shubhamugare commented 4 months ago

Exposing SyncodeLogitsProcessor through syncode. This allows users to use SynCode with any HuggingFace model simply by importing 

from syncode import SyncodeLogitsProcessor

and then providing the SyncodeLogitsProcessor in the generate method in the HF transformer model.

Check notebooks/example_logits_processor.ipynb for its use