Better External Abseil and Protobuf Linkage Support

google / sentencepiece

Unsupervised text tokenizer for Neural Network-based text generation.

Apache License 2.0

10.25k stars 1.17k forks source link

Better External Abseil and Protobuf Linkage Support #920

Closed Halmoni100 closed 9 months ago

Halmoni100 commented 1 year ago

Code changes to assist with integrating sentencepiece in cpp projects as a shared library linked to google open source dependencies externally built as shared libraries.

I tried to follow what seemed like google open source cmake conventions for the cmake options.

Code changes include:

Being more careful with isolating namespaces, include paths, and potential global namespace conflicts
Using externtal absl headers in place of internal third_party headers
Change minloglevel ABSL_FLAG to spm_minloglevel to avoid global namespace minloglevel in grpc

google-cla[bot] commented 1 year ago

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Halmoni100 commented 1 year ago

Also I kept the internal implementation of absl random because that seemed significant, I'm not really sure what the plan is there

taku910 commented 9 months ago

Sorry for the late reply. We've manually and partially merged this PR.

Due to the restrictions of the internal repositories, we don't change the directory of abseil but to make the symbolic link. In addition, abseil-cpp modules are automatically downloaded -DSPM_ABSL_PROVIDER=module is specified.