issues
search
daulet
/
tokenizers
Go bindings for HuggingFace Tokenizer
MIT License
89
stars
23
forks
source link
Bazel support
#12
Closed
daulet
closed
1 year ago
daulet
commented
1 year ago
Update core
tokenizers
to latest v0.14.1, which accidentally allows us to loose
openssl
dependency which is annoying to build in
bazel
;
Build the whole project with
bazel
, both Go and Rust part;
Check in
Cargo.lock
based on
this documentation
;
Add CI to validate everything with new build process;
Pure
cargo build
and
go build
will continue working;
tokenizers
to latest v0.14.1, which accidentally allows us to looseopenssl
dependency which is annoying to build inbazel
;bazel
, both Go and Rust part;Cargo.lock
based on this documentation;cargo build
andgo build
will continue working;