search

mistydemeo / tigerbrew

Experimental fork of homebrew for PPC Macs on Tiger
Other
536 stars 129 forks source link

How does the CFLAGS magic work? (had to kludge pcre) #689

Closed cellularmitosis closed 4 years ago

cellularmitosis commented 4 years ago

I've noticed that on most formulae, the tigerbrew set of CFLAGS seems to magically make its way into the build. Specifically, -Os -w -pipe -mcpu=750 -faltivec -mmacosx-version-min=10.4 (on my G5).

However, I noticed that pcre wasn't behaving that way. ps aux | grep CC revealed it was using -O2, for example.

I grepped through the Homebrew sources, and the only obvious hit I found was in rbconfig.rb:

macuser@imacg5$ pwd
/usr/local/Library/Homebrew/vendor/portable-ruby/2.3.3/lib/ruby/2.3.0/powerpc-darwin8.0
macuser@imacg5$ grep -- '-Os' rbconfig.rb 
  CONFIG["configure_args"] = " '--prefix=@@HOMEBREW_CELLAR@@/portable-ruby/2.3.3' '--enable-load-relative' '--with-static-linked-ext' '--without-ext=tk,sdbm,gdbm,dbm,dl,fiddle' '--disable-install-doc' '--disable-install-rdoc' '--disable-dtrace' '--with-arch=ppc' '--with-opt-dir=@@HOMEBREW_PREFIX@@/opt/portable-readline:@@HOMEBREW_PREFIX@@/opt/portable-libyaml:@@HOMEBREW_PREFIX@@/opt/portable-openssl' 'CC=cc' 'CFLAGS=-Os -w -pipe -mcpu=750 -faltivec -mmacosx-version-min=10.4' 'LDFLAGS=-L@@HOMEBREW_PREFIX@@/opt/portable-openssl/lib -L@@HOMEBREW_PREFIX@@/opt/portable-libyaml/lib -L@@HOMEBREW_PREFIX@@/opt/portable-readline/lib -L@@HOMEBREW_PREFIX@@/lib -F@@HOMEBREW_PREFIX@@/Frameworks -Wl,-headerpad_max_install_names -Wl,-search_paths_first' 'CPPFLAGS=-I@@HOMEBREW_PREFIX@@/opt/portable-openssl/include -I@@HOMEBREW_PREFIX@@/opt/portable-libyaml/include -I@@HOMEBREW_PREFIX@@/opt/portable-readline/include -F@@HOMEBREW_PREFIX@@/Frameworks' 'CXX=c++' 'CXXFLAGS=-Os -w -pipe -mcpu=750 -faltivec -mmacosx-version-min=10.4'"
  CONFIG["CXXFLAGS"] = "-Os -w -pipe -mcpu=750 -faltivec -mmacosx-version-min=10.4"
  CONFIG["CFLAGS"] = "-Os -w -pipe -mcpu=750 -faltivec -mmacosx-version-min=10.4"

I was able to modify the pcre formula to use env CFLAGS=... ./configure, but I'm unsure if this is legit, or a kludge?

  def install
    args = %W[
      CFLAGS=#{RbConfig::CONFIG["CFLAGS"]}
      ./configure
      --disable-dependency-tracking
      --prefix=#{prefix}
      --enable-utf8
      --enable-pcre8
      --enable-pcre16
      --enable-pcre32
      --enable-unicode-properties
      --enable-pcregrep-libz
      --enable-pcregrep-libbz2
    ]
    args << "--enable-jit" if MacOS.version >= :sierra

    system "./autogen.sh" if build.head?
    system "env", *args
    system "make"
cellularmitosis commented 4 years ago 
pcre-8.44 configuration summary:

    Install prefix .................. : /usr/local/Cellar/pcre/8.44_100
    C preprocessor .................. : gcc-4.2 -E
    C compiler ...................... : gcc-4.2
    C++ preprocessor ................ : g++-4.2 -E
    C++ compiler .................... : g++-4.2
    Linker .......................... : /usr/libexec/gcc/powerpc-apple-darwin9/4.2.1/ld
    C preprocessor flags ............ : 
    C compiler flags ................ : -D_THREAD_SAFE -pthread -Os -w -pipe -mcpu=750 -faltivec -mmacosx-version-min=10.4 -fvisibility=hidden
    C++ compiler flags .............. : -O2 -fvisibility=hidden -fvisibility-inlines-hidden
    Linker flags .................... : 
    Extra libraries ................. :
cellularmitosis commented 4 years ago

Oops, just realized rbconfig has -mcpu=750 not only on G3 but also on G5, etc, so this isn't a solution.

cellularmitosis commented 4 years ago

Hmm, so if I revert the formula back to having no CFLAGS shenanigans and just let brew do its thing, this is what I'm seeing:

Leopard/G5:

pcre-8.44 configuration summary:

    Install prefix .................. : /usr/local/Cellar/pcre/8.44_100
    C preprocessor .................. : gcc-4.2 -E
    C compiler ...................... : gcc-4.2
    C++ preprocessor ................ : g++-4.2 -E
    C++ compiler .................... : g++-4.2
    Linker .......................... : /usr/libexec/gcc/powerpc-apple-darwin9/4.2.1/ld
    C preprocessor flags ............ : 
    C compiler flags ................ : -g -O2 -fvisibility=hidden
    C++ compiler flags .............. : -O2 -fvisibility=hidden -fvisibility-inlines-hidden
    Linker flags .................... :

Tiger/G4e:

pcre-8.44 configuration summary:

    Install prefix .................. : /usr/local/Cellar/pcre/8.44_100
    C preprocessor .................. : /usr/local/bin/gcc-4.2 -E
    C compiler ...................... : /usr/local/bin/gcc-4.2
    C++ preprocessor ................ : /usr/local/bin/g++-4.2 -E
    C++ compiler .................... : /usr/local/bin/g++-4.2
    Linker .......................... : /usr/bin/ld
    C preprocessor flags ............ : 
    C compiler flags ................ : -Os -w -pipe -mcpu=7450 -faltivec -mmacosx-version-min=10.4 -fvisibility=hidden
    C++ compiler flags .............. : -Os -w -pipe -mcpu=7450 -faltivec -mmacosx-version-min=10.4 -fvisibility=hidden -fvisibility-inlines-hidden
    Linker flags .................... : -L/usr/local/lib -Wl,-headerpad_max_install_names

Tiger/G3:

pcre-8.44 configuration summary:

    Install prefix .................. : /usr/local/Cellar/pcre/8.44_100
    C preprocessor .................. : /usr/local/bin/gcc-4.2 -E
    C compiler ...................... : /usr/local/bin/gcc-4.2
    C++ preprocessor ................ : /usr/local/bin/g++-4.2 -E
    C++ compiler .................... : /usr/local/bin/g++-4.2
    Linker .......................... : /usr/bin/ld
    C preprocessor flags ............ : 
    C compiler flags ................ : -Os -w -pipe -mcpu=750 -faltivec -mmacosx-version-min=10.4 -fvisibility=hidden
    C++ compiler flags .............. : -Os -w -pipe -mcpu=750 -faltivec -mmacosx-version-min=10.4 -fvisibility=hidden -fvisibility-inlines-hidden
    Linker flags .................... : -L/usr/local/lib -Wl,-headerpad_max_install_names
mistydemeo commented 4 years ago

Good question! rbconfig.rb is a red herring; it's actually part of Ruby itself, and reflects what flags were used to build Ruby.

On 10.4, it's set by methods in this file: https://github.com/mistydemeo/tigerbrew/blob/master/Library/Homebrew/extend/ENV/std.rb setup_build_environment has the core logic. You can also find the CPU-specific optimization flags here: https://github.com/mistydemeo/tigerbrew/blob/master/Library/Homebrew/os/mac/hardware.rb#L5-L12

Individual packages can opt into additional options either by calling certain special helpers (for example, ENV.O2) or by directly manipulating CFLAGS via methods like ENV.append_to_cflags and ENV.append_to_ldflags.

10.5 and later use a more complex build environment tool called "superenv", which has better control over which flags get dispatched. That ensures that Tigerbrew's flags get picked up even by tools that normally ignore them.

cellularmitosis commented 4 years ago

Thanks so much!

cellularmitosis commented 4 years ago

Ok, I think something is definitely up with Leopard/G5. I noticed readline was building with -g -O, which seemed odd (Tiger/G4e and Tiger/G3 built with the expected flags).

I so downloaded the readline source tarball and ran a vanilla ./configure && make, and sure enough the default is -g -O if CFLAGS isn't set at all (confirmed this on Leopard and Tiger).

So I made a test formula which simply dumps the env and then fails: https://github.com/ppcbrew/homebrew-tiger/commit/fc75d4c7f014afb68d996b0e790ae550b8ad7324

I compared the differences and confirmed that Leopard/G5 is not setting any of the expected *FLAGS env vars.

Leopard/G5:

==> env
MANPATH=/usr/share/man:/usr/local/share/man:/usr/X11/man
HOMEBREW_USER_AGENT=Homebrew/0.9.5 (Macintosh; powerpc Mac OS X 10.5.8)
HOMEBREW_LIBRARY=/usr/local/Library
TERM=xterm-color
SHELL=/bin/bash
HOMEBREW_OS_VERSION=Mac OS X 10.5.8
HOMEBREW_REPOSITORY=/usr/local
SSH_CLIENT=192.168.4.102 47162 22
HOMEBREW_RUBY_PATH=/usr/local/Library/Homebrew/vendor/portable-ruby/current/bin/ruby
SSH_TTY=/dev/ttys002
LC_ALL=en_US.UTF-8
USER=macuser
HOMEBREW_CACHE=/Users/macuser/Library/Caches/Homebrew
HOMEBREW_USER_AGENT_CURL=Homebrew/0.9.5 (Macintosh; powerpc Mac OS X 10.5.8) curl/7.58.0
MAIL=/var/mail/macuser
PATH=/usr/local/Library/ENV/4.3:/usr/bin:/bin:/usr/sbin:/sbin
HOMEBREW_BREW_FILE=/usr/local/bin/brew
PWD=/usr/local/Library/Taps/ppcbrew/homebrew-tiger/Formula
HOMEBREW_OSX_VERSION=10.5.8
SHLVL=2
HOME=/private/tmp/cflags20200531-38867-8v5nwq/lzo-2.10/.brew_home
HOMEBREW_PREFIX=/usr/local
LOGNAME=macuser
SSH_CONNECTION=192.168.4.102 47162 192.168.4.193 22
HOMEBREW_VERSION=0.9.5
HOMEBREW_CELLAR=/usr/local/Cellar
HOMEBREW_CURL=/usr/local/Library/Homebrew/vendor/portable-curl/current/bin/curl
HOMEBREW_SYSTEM=Macintosh
HOMEBREW_ERROR_PIPE=9
OBJC=gcc-4.2
CC=gcc-4.2
HOMEBREW_CC=gcc-4.2
OBJCXX=g++-4.2
CXX=g++-4.2
HOMEBREW_CXX=g++-4.2
MAKEFLAGS=-j1
PKG_CONFIG_LIBDIR=/usr/lib/pkgconfig:/usr/local/Library/ENV/pkgconfig/10.5
HOMEBREW_CCCFG=
HOMEBREW_OPTIMIZATION_LEVEL=Os
HOMEBREW_TEMP=/tmp
HOMEBREW_OPTFLAGS=-mcpu=970
HOMEBREW_ARCHFLAGS=
CMAKE_PREFIX_PATH=/usr/local
CMAKE_INCLUDE_PATH=/usr/include/libxml2:/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers
CMAKE_LIBRARY_PATH=/System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries
ACLOCAL_PATH=/usr/local/share/aclocal
HOMEBREW_ISYSTEM_PATHS=/usr/local/include:/usr/include/libxml2:/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers
HOMEBREW_LIBRARY_PATHS=/usr/local/lib:/System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries
HOMEBREW_CC_LOG_PATH=/Users/macuser/Library/Logs/Homebrew/cflags/01.env

Tiger/G4e:

==> env
HOMEBREW_USER_AGENT=Homebrew/0.9.5 (Macintosh; powerpc Mac OS X 10.4.11)
HOMEBREW_LIBRARY=/usr/local/Library
TERM=xterm-color
SHELL=/bin/bash
HOMEBREW_OS_VERSION=Mac OS X 10.4.11
HOMEBREW_REPOSITORY=/usr/local
SSH_CLIENT=192.168.4.102 41286 22
HOMEBREW_RUBY_PATH=/usr/local/Library/Homebrew/vendor/portable-ruby/current/bin/ruby
SSH_TTY=/dev/ttyp4
LC_ALL=en_US.UTF-8
USER=macuser
HOMEBREW_CACHE=/Users/macuser/Library/Caches/Homebrew
HOMEBREW_USER_AGENT_CURL=Homebrew/0.9.5 (Macintosh; powerpc Mac OS X 10.4.11) curl/7.58.0
MAIL=/var/mail/macuser
PATH=/Users/macuser/bin:/usr/local/Library/Taps/ppcbrew/homebrew-tiger/bin:usr/local/sbin:/usr/local/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/Library/ENV/scm
HOMEBREW_BREW_FILE=/usr/local/bin/brew
PWD=/usr/local/Library/Taps/ppcbrew/homebrew-tiger/Formula
HOMEBREW_OSX_VERSION=10.4.11
SHLVL=2
HOME=/private/tmp/cflags20200531-27683-1utp602/lzo-2.10/.brew_home
HOMEBREW_PREFIX=/usr/local
LOGNAME=macuser
SSH_CONNECTION=192.168.4.102 41286 192.168.4.191 22
HOMEBREW_VERSION=0.9.5
HOMEBREW_CELLAR=/usr/local/Cellar
HOMEBREW_CURL=/usr/local/Library/Homebrew/vendor/portable-curl/current/bin/curl
HOMEBREW_SYSTEM=Macintosh
__CF_USER_TEXT_ENCODING=0x1F5:0:0
HOMEBREW_ERROR_PIPE=10
PKG_CONFIG_LIBDIR=/usr/local/lib/pkgconfig:/usr/local/Library/ENV/pkgconfig/10.4:/usr/lib/pkgconfig
MAKEFLAGS=-j1
CPPFLAGS=-F/usr/local/Frameworks
LDFLAGS=-L/usr/local/lib -F/usr/local/Frameworks -Wl,-headerpad_max_install_names
CMAKE_FRAMEWORK_PATH=/usr/local/Frameworks
CFLAGS=-Os -w -pipe -mcpu=7450 -faltivec -mmacosx-version-min=10.4
CXXFLAGS=-Os -w -pipe -mcpu=7450 -faltivec -mmacosx-version-min=10.4
OBJCFLAGS=-Os -w -pipe -mcpu=7450 -faltivec -mmacosx-version-min=10.4
OBJCXXFLAGS=-Os -w -pipe -mcpu=7450 -faltivec -mmacosx-version-min=10.4
OBJC=/usr/local/bin/gcc-4.2
CC=/usr/local/bin/gcc-4.2
OBJCXX=/usr/local/bin/g++-4.2
CXX=/usr/local/bin/g++-4.2
MACOSX_DEPLOYMENT_TARGET=10.4
CPATH=/usr/local/include
HOMEBREW_CC_LOG_PATH=/Users/macuser/Library/Logs/Homebrew/cflags/01.env

Here's which env keys are present/missing on Tiger/G4e vs Leopard/G5:

$ diff -urN g5.keys g4e.keys 
--- g5.keys 2020-05-31 05:51:01.599474461 -0500
+++ g4e.keys    2020-05-31 05:51:09.859723403 -0500
@@ -1,41 +1,37 @@
-ACLOCAL_PATH
 CC
-CMAKE_INCLUDE_PATH
-CMAKE_LIBRARY_PATH
-CMAKE_PREFIX_PATH
+CFLAGS
+__CF_USER_TEXT_ENCODING
+CMAKE_FRAMEWORK_PATH
+CPATH
+CPPFLAGS
+CXXFLAGS
 CXX
-HOMEBREW_ARCHFLAGS
 HOMEBREW_BREW_FILE
 HOMEBREW_CACHE
-HOMEBREW_CCCFG
-HOMEBREW_CC
 HOMEBREW_CC_LOG_PATH
 HOMEBREW_CELLAR
 HOMEBREW_CURL
-HOMEBREW_CXX
 HOMEBREW_ERROR_PIPE
-HOMEBREW_ISYSTEM_PATHS
-HOMEBREW_LIBRARY_PATHS
 HOMEBREW_LIBRARY
-HOMEBREW_OPTFLAGS
-HOMEBREW_OPTIMIZATION_LEVEL
 HOMEBREW_OS_VERSION
 HOMEBREW_OSX_VERSION
 HOMEBREW_PREFIX
 HOMEBREW_REPOSITORY
 HOMEBREW_RUBY_PATH
 HOMEBREW_SYSTEM
-HOMEBREW_TEMP
 HOMEBREW_USER_AGENT_CURL
 HOMEBREW_USER_AGENT
 HOMEBREW_VERSION
 HOME
 LC_ALL
+LDFLAGS
 LOGNAME
+MACOSX_DEPLOYMENT_TARGET
 MAIL
 MAKEFLAGS
-MANPATH
+OBJCFLAGS
 OBJC
+OBJCXXFLAGS
 OBJCXX
 PATH
 PKG_CONFIG_LIBDIR

I'll try to take a closer look at ENV/std.rb to see if I can figure this out.

cellularmitosis commented 4 years ago

It looks like this comes down to different implementations of setup_build_environment getting called. It looks like Leopard is ENV/super.rb while Tiger is ENV/std.rb?

cellularmitosis commented 4 years ago

oh, lol, I just discovered brew --env:

Leopard/G5:

$ brew --env
HOMEBREW_CC: gcc-4.2
HOMEBREW_CXX: g++-4.2
MAKEFLAGS: -j1
CMAKE_PREFIX_PATH: /usr/local
CMAKE_INCLUDE_PATH: /usr/include/libxml2:/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers
CMAKE_LIBRARY_PATH: /System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries
PKG_CONFIG_LIBDIR: /usr/lib/pkgconfig:/usr/local/Library/ENV/pkgconfig/10.5
ACLOCAL_PATH: /usr/local/share/aclocal
PATH: /usr/local/Library/ENV/4.3:/usr/bin:/bin:/usr/sbin:/sbin

Tiger/G3:

$ brew --env
CC: /usr/local/bin/gcc-4.2 => /usr/local/Cellar/apple-gcc42/4.2.1-5553/bin/gcc-4.2
CXX: /usr/local/bin/g++-4.2 => /usr/local/Cellar/apple-gcc42/4.2.1-5553/bin/g++-4.2
OBJC: /usr/local/bin/gcc-4.2
OBJCXX: /usr/local/bin/g++-4.2
CFLAGS: -Os -w -pipe -mcpu=750 -faltivec -mmacosx-version-min=10.4
CXXFLAGS: -Os -w -pipe -mcpu=750 -faltivec -mmacosx-version-min=10.4
CPPFLAGS: -F/usr/local/Frameworks
LDFLAGS: -L/usr/local/lib -F/usr/local/Frameworks -Wl,-headerpad_max_install_names
MAKEFLAGS: -j1
CMAKE_FRAMEWORK_PATH: /usr/local/Frameworks
MACOSX_DEPLOYMENT_TARGET: 10.4
PKG_CONFIG_LIBDIR: /usr/local/lib/pkgconfig:/usr/local/Library/ENV/pkgconfig/10.4:/usr/lib/pkgconfig
PATH: /Users/macuser/bin:/usr/local/sbin:/usr/local/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/Library/ENV/scm
CPATH: /usr/local/include
cellularmitosis commented 4 years ago

Ok, so Tiger uses Xcode tools 2.5, which has no entry in /usr/local/Library/ENV/*, so it becomes a std env.

Leopard uses Xcode tools 3.1.4, which does have an entry in /usr/local/Library/ENV/*, so it becomes a super env.

If I want Leopard to behave like a std env, it looks like I can use brew install --env=std foo.

aha, yes, that works.

Example compilation line from Tiger/G3 building readline:

/usr/local/bin/gcc-4.2 -c -DHAVE_CONFIG_H   -I. -I.. -I.. -F/usr/local/Frameworks -DRL_LIBRARY_VERSION='"8.0"' -Os -w -pipe -mcpu=750 -faltivec -mmacosx-version-min=10.4 -fno-common -o kill.o ../kill.c

Default behavior of Leopard/G5 building readline:

gcc-4.2 -c -DHAVE_CONFIG_H   -I. -I.. -I..  -DRL_LIBRARY_VERSION='"8.0"' -g -O -fno-common -o kill.o ../kill.c

Vanilla ./configure && make of readline tarball:

gcc -c  -DHAVE_CONFIG_H   -I. -I.  -DRL_LIBRARY_VERSION='"8.0"' -g -O kill.c

Leopard/G5 invoked via brew install --env=std readline:

/usr/bin/gcc-4.2 -c  -DHAVE_CONFIG_H   -I. -I. -F/usr/local/Frameworks -DRL_LIBRARY_VERSION='"8.0"' -Os -w -pipe -mcpu=970 -mmacosx-version-min=10.5 kill.c