google / sentencepiece

Unsupervised text tokenizer for Neural Network-based text generation.
Apache License 2.0
10.25k stars 1.17k forks source link

Additional external absl fixes #962

Closed Halmoni100 closed 9 months ago

Halmoni100 commented 9 months ago

The last merge was good, but there are a few new issues.

The first is straightforward. Rerunning cmake returns an error from the file command. This is fixed with the first commit.

The second is an issue with the LOG and CHECK macros. Somehow bringing in external absl headers must include its own definitions for LOG and CHECK that override the sentencepiece definitions. So I added the corresponding libraries to use the external absl macros. I'm not sure this is desired behavior, but it fixes the build issue.