search

CdC-SI / eak-copilot

The official repository of the EAK-Copilot project as part of the Innovation Fellowship 2024.
https://cdc-si.github.io/eak-copilot/
GNU General Public License v3.0
4 stars 0 forks source link

create tokenizer components and factory #254

Open K-Schubert opened 1 week ago

K-Schubert commented 1 week ago

Description

Need to create and extend the TokenizerFactory class to accept tokenizers for open-source embedding models.