On an arm64 Mac, brew appears to install a broken version of gettext

ndw commented 1 year ago

`brew gist-logs <formula>` link OR `brew config` AND `brew doctor` output

$ brew config
HOMEBREW_VERSION: 3.6.15
ORIGIN: https://github.com/Homebrew/brew
HEAD: 11cdffb4fef84828a7e02860212b30ce4fac85f5
Last commit: 5 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 7880424a9f86170a7d85dbd671d584904ea5c2d3
Core tap last commit: 10 minutes ago
Core tap branch: master
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: emacs
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.10 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 14.0.0 build 1400
Git: 2.39.0 => /opt/homebrew/bin/git
Curl: 7.79.1 => /usr/bin/curl
macOS: 12.6.2-arm64
CLT: 14.2.0.0.1.1668646533
Xcode: 14.2
Rosetta 2: false

and

$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: Unbrewed dylibs were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected dylibs:
  /usr/local/lib/libguile-mu.0.dylib

Warning: Unbrewed '.la' files were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected '.la' files:
  /usr/local/lib/libguile-mu.la

Verification

[X] I ran brew update and am still able to reproduce my issue.
[X] I have resolved all warnings from brew doctor and that did not fix my problem.

What were you trying to do (and why)?

I want to build Python (3.11.1). I'm trying to setup virtual environments, when that didn't work, I took a step back to see if I could simply compile python.

(I didn't actually delete the libguile files identified by brew doctor but I don't think they're involved.)

What happened (include all command output)?

Make fails:

gcc     -o Programs/_freeze_module Programs/_freeze_module.o Modules/getpath_noop.o Modules/getbuildinfo.o Parser/token.o  Parser/pegen.o Parser/pegen_errors.o Parser/action_helpers.o Parser/parser.o Parser/string_parser.o Parser/peg_api.o Parser/myreadline.o Parser/tokenizer.o Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/call.o Objects/capsule.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genericaliasobject.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/interpreteridobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/odictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/picklebufobject.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/unionobject.o Objects/weakrefobject.o Python/_warnings.o Python/Python-ast.o Python/Python-tokenize.o Python/asdl.o Python/ast.o Python/ast_opt.o Python/ast_unparse.o Python/bltinmodule.o Python/ceval.o Python/codecs.o Python/compile.o Python/context.o Python/dynamic_annotations.o Python/errors.o Python/frame.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/hamt.o Python/hashtable.o Python/import.o Python/importdl.o Python/initconfig.o Python/marshal.o Python/modsupport.o Python/mysnprintf.o Python/mystrtoul.o Python/pathconfig.o Python/preconfig.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pylifecycle.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/pytime.o Python/bootstrap_hash.o Python/specialize.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/thread.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/pystrhex.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/suggestions.o Python/dynload_shlib.o     Modules/config.o Modules/main.o Modules/gcmodule.o Modules/atexitmodule.o  Modules/faulthandler.o  Modules/posixmodule.o  Modules/signalmodule.o  Modules/_tracemalloc.o  Modules/_codecsmodule.o  Modules/_collectionsmodule.o  Modules/errnomodule.o  Modules/_io/_iomodule.o Modules/_io/iobase.o Modules/_io/fileio.o Modules/_io/bytesio.o Modules/_io/bufferedio.o Modules/_io/textio.o Modules/_io/stringio.o  Modules/itertoolsmodule.o  Modules/_sre/sre.o  Modules/_threadmodule.o  Modules/timemodule.o  Modules/_weakref.o  Modules/_abc.o  Modules/_functoolsmodule.o  Modules/_localemodule.o  Modules/_operator.o  Modules/_stat.o  Modules/symtablemodule.o  Modules/pwdmodule.o  Modules/xxsubtype.o -ldl  -framework CoreFoundation
Undefined symbols for architecture arm64:
  "_libintl_bindtextdomain", referenced from:
      __locale_bindtextdomain in _localemodule.o
  "_libintl_dcgettext", referenced from:
      __locale_dcgettext in _localemodule.o
  "_libintl_dgettext", referenced from:
      __locale_dgettext in _localemodule.o
  "_libintl_gettext", referenced from:
      __locale_gettext in _localemodule.o
  "_libintl_setlocale", referenced from:
      __locale_setlocale in _localemodule.o
      __locale_localeconv in _localemodule.o
  "_libintl_textdomain", referenced from:
      __locale_textdomain in _localemodule.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
gmake: *** [Makefile:1188: Programs/_freeze_module] Error 1

A little web searching lead me to some suggestions about removing gettext. I had trouble with that because of other installed packages with dependencies on gettext, so I just uninstalled Homebrew. Then I reinstalled it, but didn't install any packages.

In this state, Python builds fine.

Then I installed git which I need, and make because I have an alias for make that expects gmake to exist. And I installed exa because I have an alias for ls that uses it. I don't have any reason to think make and exa are involved anyway.

$ brew list
==> Formulae
exa gettext git make    pcre2

In this state, Python won't build with the error above.

I don't see any obvious problem with the gettext libraries:

$ for f in `find /opt/homebrew -type f -print|grep gettext|grep dylib`; do file $f; done
/opt/homebrew/Cellar/gettext/0.21.1/lib/libgettextpo.0.dylib: Mach-O 64-bit dynamically linked shared library arm64
/opt/homebrew/Cellar/gettext/0.21.1/lib/libgettextlib-0.21.1.dylib: Mach-O 64-bit dynamically linked shared library arm64
/opt/homebrew/Cellar/gettext/0.21.1/lib/libtextstyle.0.dylib: Mach-O 64-bit dynamically linked shared library arm64
/opt/homebrew/Cellar/gettext/0.21.1/lib/libintl.8.dylib: Mach-O 64-bit dynamically linked shared library arm64
/opt/homebrew/Cellar/gettext/0.21.1/lib/libasprintf.0.dylib: Mach-O 64-bit dynamically linked shared library arm64
/opt/homebrew/Cellar/gettext/0.21.1/lib/libgettextsrc-0.21.1.dylib: Mach-O 64-bit dynamically linked shared library arm64

But I don't really know what else to try.

What did you expect to happen?

I expected Python to build :-)

Step-by-step reproduction instructions (by running `brew` commands)

See above.

cho-m commented 1 year ago

I don't think gettext is broken.

If I had to guess, Python's build is detecting libintl but not using necessary linker flags when compiling.

On GNU/Linux, libintl is provided by glibc, so it usually doesn't require additional linker flags.
On macOS (and I think various BSD), libintl is from external gettext, so it requires passing -lintl to link to corresponding library.

From your snippet, the failing step is only passing: -ldl -framework CoreFoundation so it is missing -lintl. You may need to review upstream build scripts to see why. Usually cross-platform autotools configure script should detect if additional linker flags are needed and add those into Makefile steps as needed.

The symbols should all be available once -lintl is passed and corresponding libintl.dylib is found:

❯ nm -gU /opt/homebrew/opt/gettext/lib/libintl.dylib | grep libintl
000000000000449c T _libintl_bind_textdomain_codeset
000000000000417c T _libintl_bindtextdomain
00000000000044c8 T _libintl_dcgettext
0000000000007994 T _libintl_dcngettext
00000000000044dc T _libintl_dgettext
00000000000079a4 T _libintl_dngettext
00000000000044e4 T _libintl_gettext
0000000000009540 T _libintl_newlocale
00000000000079ac T _libintl_ngettext
00000000000083e8 T _libintl_set_relocation_prefix
0000000000009048 T _libintl_setlocale
0000000000006100 T _libintl_textdomain
00000000000103e0 D _libintl_version

cho-m commented 1 year ago

Also, since some brew doctor output mentions /usr/local/lib, if you have a Rosetta/x86_64 installation of Homebrew in /usr/local (or files left over from migrating a Intel mac to ARM mac), you may want to review what libraries are installed there.

/usr/local is a standard path that is often picked up by compilers. This can cause issues if you mix incompatible architecture libraries.

carlocab commented 1 year ago

This is the first I'm hearing of this, so it seems unlikely that the issue is with Homebrew gettext. I am willing to revise this belief given the right evidence, of course.

fxcoudert commented 1 year ago

From your snippet, the failing step is only passing: -ldl -framework CoreFoundation so it is missing -lintl

Exactly this. The error is in the build process.

ndw commented 1 year ago

Indeed, it builds fine with

make LDFLAGS="-L/opt/homebrew/opt/gettext/lib -lintl"

Thank you for your patience. Now, can I persuade pyenv to do this? We shall see...

Homebrew / homebrew-core