Closed TheEssem closed 2 weeks ago
@TheEssem In case you're encountering this while running Mastodon on a system with ICU 75, forking this library to remove transliterator.cpp
(and its reference in ext.c
) entirely fixes the issue temporarily until this library is updated.
Does using BUNDLE_BUILD__CHARLOCK_HOLMES="--with-cxxflags=-std=c++17"
solve the issue for you, using the latest 0.7.8 charlock_holmes
gem?
I just ran into this error and the fix mentioned by @renchap did the trick.
Just doing this failed with the error in the original issue:
bundle install
But this worked:
BUNDLE_BUILD__CHARLOCK_HOLMES="--with-cxxflags=-std=c++17" bundle install
Actually, after looking back at the error I was getting I'm not sure that it's exactly the same one. Here's what I was seeing, just in case it helps anybody else find this issue/fix:
Installing charlock_holmes 0.7.8 with native extensions
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory: /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/charlock_holmes-0.7.8/ext/charlock_holmes
/Users/jgreen/.asdf/installs/ruby/3.3.2/bin/ruby extconf.rb
checking for pkg-config for icu-i18n... ["-I/opt/homebrew/Cellar/icu4c/74.2/include ", "-L/opt/homebrew/Cellar/icu4c/74.2/lib", "-licui18n -licuuc
-licudata"]
checking for pkg-config for icu-io... ["-I/opt/homebrew/Cellar/icu4c/74.2/include ", "-L/opt/homebrew/Cellar/icu4c/74.2/lib", "-licuio -licui18n -licuuc
-licudata"]
checking for pkg-config for icu-uc... ["-I/opt/homebrew/Cellar/icu4c/74.2/include ", "-L/opt/homebrew/Cellar/icu4c/74.2/lib", "-licuuc -licudata"]
checking for -licui18n... yes
checking for unicode/ucnv.h... yes
checking for -lz... yes
checking for -licuuc... yes
checking for -licudata... yes
creating Makefile
current directory: /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/charlock_holmes-0.7.8/ext/charlock_holmes
make DESTDIR\= sitearchdir\=./.gem.20240701-74827-mdklv sitelibdir\=./.gem.20240701-74827-mdklv clean
current directory: /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/charlock_holmes-0.7.8/ext/charlock_holmes
make DESTDIR\= sitearchdir\=./.gem.20240701-74827-mdklv sitelibdir\=./.gem.20240701-74827-mdklv
compiling converter.c
converter.c:23:12: warning: implicit conversion loses integer precision: 'long' to 'int32_t' (aka 'int') [-Wshorten-64-to-32]
src_len = RSTRING_LEN(rb_txt);
~ ^~~~~~~~~~~~~~~~~~~
/Users/jgreen/.asdf/installs/ruby/3.3.2/include/ruby-3.3.0/ruby/internal/core/rstring.h:46:27: note: expanded from macro 'RSTRING_LEN'
#define RSTRING_LEN RSTRING_LEN
^
1 warning generated.
compiling encoding_detector.c
encoding_detector.c:183:71: warning: implicit conversion loses integer precision: 'long' to 'int32_t' (aka 'int') [-Wshorten-64-to-32]
ucsdet_setDeclaredEncoding(detector->csd, RSTRING_PTR(rb_enc_hint), RSTRING_LEN(rb_enc_hint), &status);
~~~~~~~~~~~~~~~~~~~~~~~~~~ ^~~~~~~~~~~~~~~~~~~~~~~~
/Users/jgreen/.asdf/installs/ruby/3.3.2/include/ruby-3.3.0/ruby/internal/core/rstring.h:46:27: note: expanded from macro 'RSTRING_LEN'
#define RSTRING_LEN RSTRING_LEN
^
encoding_detector.c:232:71: warning: implicit conversion loses integer precision: 'long' to 'int32_t' (aka 'int') [-Wshorten-64-to-32]
ucsdet_setDeclaredEncoding(detector->csd, RSTRING_PTR(rb_enc_hint), RSTRING_LEN(rb_enc_hint), &status);
~~~~~~~~~~~~~~~~~~~~~~~~~~ ^~~~~~~~~~~~~~~~~~~~~~~~
/Users/jgreen/.asdf/installs/ruby/3.3.2/include/ruby-3.3.0/ruby/internal/core/rstring.h:46:27: note: expanded from macro 'RSTRING_LEN'
#define RSTRING_LEN RSTRING_LEN
^
2 warnings generated.
compiling ext.c
compiling transliterator.cpp
In file included from transliterator.cpp:5:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/translit.h:26:
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/uobject.h:134:54: error: expected ';' at end of declaration list
static void * U_EXPORT2 operator new(size_t size) noexcept;
^
;
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/uobject.h:141:56: error: expected ';' at end of declaration list
static void * U_EXPORT2 operator new[](size_t size) noexcept;
^
;
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/uobject.h:151:51: error: expected ';' at end of declaration list
static void U_EXPORT2 operator delete(void *p) noexcept;
^
;
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/uobject.h:158:53: error: expected ';' at end of declaration list
static void U_EXPORT2 operator delete[](void *p) noexcept;
^
;
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/uobject.h:166:67: error: expected ';' at end of declaration list
static inline void * U_EXPORT2 operator new(size_t, void *ptr) noexcept { return ptr; }
^
;
In file included from transliterator.cpp:5:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/translit.h:27:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:36:
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/char16ptr.h:42:30: warning: 'final' keyword is a C++11 extension [-Wc++11-extensions]
class U_COMMON_API Char16Ptr final {
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/char16ptr.h:93:19: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
Char16Ptr() = delete;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/char16ptr.h:149:35: warning: 'final' keyword is a C++11 extension [-Wc++11-extensions]
class U_COMMON_API ConstChar16Ptr final {
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/char16ptr.h:201:24: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
ConstChar16Ptr() = delete;
^
In file included from transliterator.cpp:5:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/translit.h:27:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:39:
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/stringpiece.h:133:22: warning: default template arguments for a function template are a C++11 extension
[-Wc++11-extensions]
typename = typename std::enable_if<
^ ~~~~~~~~~~~~~~~~~~~~~~~~
In file included from transliterator.cpp:5:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/translit.h:27:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:40:
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/bytestream.h:164:32: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
ByteSink(const ByteSink &) = delete;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/bytestream.h:165:43: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
ByteSink &operator=(const ByteSink &) = delete;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/bytestream.h:209:53: warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
virtual void Append(const char* bytes, int32_t n) override;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/bytestream.h:227:59: warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
int32_t* result_capacity) override;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/bytestream.h:256:28: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
CheckedArrayByteSink() = delete;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/bytestream.h:257:56: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
CheckedArrayByteSink(const CheckedArrayByteSink &) = delete;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/bytestream.h:258:67: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
CheckedArrayByteSink &operator=(const CheckedArrayByteSink &) = delete;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/bytestream.h:294:52: warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
virtual void Append(const char* data, int32_t n) override { dest_->append(data, n); }
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/bytestream.h:298:22: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
StringByteSink() = delete;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/bytestream.h:299:44: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
StringByteSink(const StringByteSink &) = delete;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/bytestream.h:300:55: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
StringByteSink &operator=(const StringByteSink &) = delete;
^
In file included from transliterator.cpp:5:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/translit.h:27:
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:1529:44: warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
UnicodeString& target) const override;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:1909:42: warning: rvalue references are a C++11 extension [-Wc++11-extensions]
UnicodeString &operator=(UnicodeString &&src) noexcept;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:1909:48: error: expected ';' at end of declaration list
UnicodeString &operator=(UnicodeString &&src) noexcept;
^
;
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:1916:34: error: expected ';' at end of declaration list
void swap(UnicodeString &other) noexcept;
^
;
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:1925:45: error: expected ';' at end of declaration list
swap(UnicodeString &s1, UnicodeString &s2) noexcept {
^
;
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:2459:64: warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
const UnicodeString& text) override;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:2466:37: warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
virtual UBool hasMetaData() const override;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:2481:65: warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
virtual void copy(int32_t start, int32_t limit, int32_t dest) override;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3309:31: warning: rvalue references are a C++11 extension [-Wc++11-extensions]
UnicodeString(UnicodeString &&src) noexcept;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3309:37: error: expected ';' at end of declaration list
UnicodeString(UnicodeString &&src) noexcept;
^
;
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3341:40: warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
virtual UnicodeString *clone() const override;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3447:46: warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
virtual UClassID getDynamicClassID() const override;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3458:37: warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
virtual int32_t getLength() const override;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3465:52: warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
virtual char16_t getCharAt(int32_t offset) const override;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3472:53: warning: 'override' keyword is a C++11 extension [-Wc++11-extensions]
virtual UChar32 getChar32At(int32_t offset) const override;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3634:63: error: expected ';' at end of declaration list
void copyFieldsFrom(UnicodeString &src, UBool setSrcToBogus) noexcept;
^
;
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:2998:7: error: delegating constructors are permitted only in C++11
UnicodeString(ConstChar16Ptr(text)) {}
^~~~~~~~~~~~~
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3048:7: error: delegating constructors are permitted only in C++11
UnicodeString(ConstChar16Ptr(text), textLength) {}
^~~~~~~~~~~~~
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3129:7: error: delegating constructors are permitted only in C++11
UnicodeString(Char16Ptr(buffer), buffLength, buffCapacity) {}
^~~~~~~~~~~~~
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:4595:16: error: out-of-line definition of 'operator=' does not match any declaration in
'icu_74::UnicodeString'
UnicodeString::operator= (char16_t ch)
^~~~~~~~
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:4742:12: error: call to member function 'doReplace' is ambiguous
return doReplace(start, _length, nullptr, 0, 0);
^~~~~~~~~
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3574:18: note: candidate function
UnicodeString& doReplace(int32_t start,
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3580:18: note: candidate function
UnicodeString& doReplace(int32_t start,
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:4748:10: error: call to member function 'doReplace' is ambiguous
{ return doReplace(start, limit - start, nullptr, 0, 0); }
^~~~~~~~~
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3574:18: note: candidate function
UnicodeString& doReplace(int32_t start,
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3580:18: note: candidate function
UnicodeString& doReplace(int32_t start,
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:4753:10: error: call to member function 'doReplace' is ambiguous
return doReplace(0, start, nullptr, 0, 0);
^~~~~~~~~
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3574:18: note: candidate function
UnicodeString& doReplace(int32_t start,
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/unistr.h:3580:18: note: candidate function
UnicodeString& doReplace(int32_t start,
^
In file included from transliterator.cpp:5:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/translit.h:29:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/utrans.h:22:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/uenum.h:25:
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/localpointer.h:71:51: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
static void* U_EXPORT2 operator new(size_t) = delete;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/localpointer.h:72:53: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
static void* U_EXPORT2 operator new[](size_t) = delete;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/localpointer.h:74:58: warning: deleted function definitions are a C++11 extension [-Wc++11-extensions]
static void* U_EXPORT2 operator new(size_t, void*) = delete;
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/localpointer.h:224:34: warning: rvalue references are a C++11 extension [-Wc++11-extensions]
LocalPointer(LocalPointer<T> &&src) noexcept : LocalPointerBase<T>(src.ptr) {
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/localpointer.h:224:40: error: expected ';' at end of declaration list
LocalPointer(LocalPointer<T> &&src) noexcept : LocalPointerBase<T>(src.ptr) {
^
;
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/localpointer.h:399:30: warning: rvalue references are a C++11 extension [-Wc++11-extensions]
LocalArray(LocalArray<T> &&src) noexcept : LocalPointerBase<T>(src.ptr) {
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/localpointer.h:399:36: error: expected ';' at end of declaration list
LocalArray(LocalArray<T> &&src) noexcept : LocalPointerBase<T>(src.ptr) {
^
;
In file included from transliterator.cpp:5:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/translit.h:29:
In file included from /opt/homebrew/Cellar/icu4c/74.2/include/unicode/utrans.h:22:
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/uenum.h:69:1: warning: rvalue references are a C++11 extension [-Wc++11-extensions]
U_DEFINE_LOCAL_OPEN_POINTER(LocalUEnumerationPointer, UEnumeration, uenum_close);
^
/opt/homebrew/Cellar/icu4c/74.2/include/unicode/localpointer.h:556:53: note: expanded from macro 'U_DEFINE_LOCAL_OPEN_POINTER'
LocalPointerClassName(LocalPointerClassName &&src) noexcept \
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
33 warnings and 20 errors generated.
make: *** [transliterator.o] Error 1
make failed, exit code 2
Gem files will remain installed in /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/charlock_holmes-0.7.8 for inspection.
Results logged to /Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/extensions/arm64-darwin-23/3.3.0/charlock_holmes-0.7.8/gem_make.out
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/ext/builder.rb:125:in `run'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/ext/builder.rb:51:in `block in make'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/ext/builder.rb:43:in `each'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/ext/builder.rb:43:in `make'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/ext/ext_conf_builder.rb:42:in `build'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/ext/builder.rb:193:in `build_extension'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/ext/builder.rb:227:in `block in build_extensions'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/ext/builder.rb:224:in `each'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/ext/builder.rb:224:in `build_extensions'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/3.3.0/rubygems/installer.rb:852:in `build_extensions'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/rubygems_gem_installer.rb:72:in `build_extensions'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/rubygems_gem_installer.rb:28:in `install'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/source/rubygems.rb:200:in `install'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/installer/gem_installer.rb:54:in `install'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/installer/gem_installer.rb:16:in `install_from_spec'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/installer/parallel_installer.rb:156:in `do_install'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/installer/parallel_installer.rb:147:in `block in worker_pool'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/worker.rb:62:in `apply_func'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/worker.rb:57:in `block in process_queue'
<internal:kernel>:187:in `loop'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/worker.rb:54:in `process_queue'
/Users/jgreen/.asdf/installs/ruby/3.3.2/lib/ruby/gems/3.3.0/gems/bundler-2.4.10/lib/bundler/worker.rb:90:in `block (2 levels) in create_threads'
An error occurred while installing charlock_holmes (0.7.8), and Bundler cannot continue.
In Gemfile:
bullet_train-incoming_webhooks was resolved to 1.7.14, which depends on
bullet_train-api was resolved to 1.7.14, which depends on
bullet_train was resolved to 1.7.14, which depends on
extended_email_reply_parser was resolved to 0.5.1, which depends on
charlock_holmes
It looks like 0.7.8 solved the issue on my end, no extra flags needed.
When attempting to build this gem on EndeavourOS Linux using Ruby version 3.3.1 and ICU version 75.1, I get the following error output:
It seems like the related upstream commits are probably unicode-org/icu@d0672fa and unicode-org/icu@6fa113e?