Add support for Llama models

CodeWithKyrian / transformers-php

Transformers PHP is a toolkit for PHP developers to add machine learning magic to their projects easily.

https://codewithkyrian.github.io/transformers-php/

Apache License 2.0

530 stars 28 forks source link

Add support for Llama models #25

Closed CodeWithKyrian closed 5 months ago

CodeWithKyrian commented 5 months ago

What:

[ ] Bug Fix
[x] New Feature

Description:

This PR adds support for Llama models and resolves a bug in the LlamaTokenizer that previously used an underscore instead of the expected Spiece underline character.