daulet / tokenizers

Go bindings for HuggingFace Tokenizer
MIT License
85 stars 23 forks source link

Add tokenizers_srcdir_relative build tag to allow static library path #15

Closed RJKeevil closed 3 months ago

RJKeevil commented 10 months ago

Add tokenizers_srcdir_relative build tag to allow library to be placed either in the module source directory, or in the static /usr/lib/ path.

Due to the suggested naming of the variable, I have therefore made the relative path option non-default. If you would prefer the current behaviour to be unchanged, should we instead make the tag tokenizers_srcdir_static or something?

jmoney commented 4 months ago

I think https://github.com/daulet/tokenizers/pull/18 solves this issue too.

daulet commented 3 months ago

I believe https://github.com/daulet/tokenizers/pull/18 solves this more generically.