Closed RandomDSdevel closed 7 years ago
Oh, wait; never mind, the old brew config
is logged at the end of 02.make
. Silly me, Trix are for kids.
OK, from what I can see at first glance, the failed line has something to do with QtWebEngine…I'll take another look at it tomorrow with fresh eyes, as mine are currently glazing over a bit. In the meantime, perhaps I should run another build attempt and actually log it this time…
All right, here are the logs I took of the second failed Qt v5.7.1 build attempt I ran last night. You'll be relieved to hear that config.out
is actually a record of the output of running brew config
before running brew update
this time. Also, I had to compress some files to make them fit again, but they're up there.
Also, I think it might be helpful for you guys to know that building QtWebEngine failed during an attempt to link something called Demobrowser.app/Contents/MacOS/Demobrowser
. I'm not sure what $PWD
was at that point, though, so I can't locate that within Qt's source right off the bat. I might try an mdfind
, though…
Aaaaannd…, nothing. I probably wasn't looking for the right thing or wasn't looking in exactly the right place, though, even though I searched inside where Homebrew stashed my failed build's leftover temporary files.
@RandomDSdevel Closing this since CI was able to build with the options listed above. See https://github.com/Homebrew/homebrew-core/pull/8523.
@ilovezfs: Huh, it's weird that building Qt only fails on my machine. I guess I'll just have to investigate this further on my own and report back if I find a fix.
Also, @ilovezfs (taken from #8523:)
So it looks like #8502 may actually be about the --with-examples option: #8502 (comment)
That's strange, as I'm pretty sure the fix we carried off and on as a result of #1948, #6766, and #6844 got merged into Qt v5.7.1 upstream, so it's most likely not that problem we're dealing with here. I'd better go ahead and report this upstream, then come back here with the resulting patch, then…
OK, I've reported my issue upstream to get some additional eyeballs on it as QTBUG-58011. Let's hope somebody can reproduce it and figure out what needs fixing…
@RandomDSdevel It's very likely they'll ask you to build from source outside of Homebrew itself. Additionally, as I've mentioned before, unless you need all the options, don't build with all the options and try pouring the bottle instead.
Well, if I have to go through the additional hassle of building Qt more than one more time, I guess I'll just have to grit my teeth and bear it. As for options and bottles, I agree that this would usually be the case despite the fact that I, personally, would normally save pouring a bottle until needed as a last resort due to my anomalous predisposition towards building everything from source, but isn't it #8523's intention to enable all options by default?
P. S.: On another note, it looks like Qt WebEngine's Agile board is currently full to bursting, so getting a response from upstream may take longer than we (or at least I) would like. Grrr, sigh, dum-de-dum…
@RandomDSdevel you could try adding ENV.deparallelize
to the formula, and see if it makes any difference.
You can
export HOMEBREW_MAKE_JOBS=1
if you don't want to edit the formula file.
@RandomDSdevel I understand why you want to build from source (despite disagreeing) but if you're filing issues about it it's worth being 100% sure they are reproducible and giving it your best shot to try and fix them yourself first.
@ilovezfs: Huh, didn't think about deparallelizing the build. Good call; I'll try that and see if the problem might have something to do with dependency ordering.
@MikeMcQuaid: The issue reappears consistently on my machine; it's just getting it to show up somewhere else that's a problem, as I don't have any other machines under my control, let alone running Homebrew installs, to work with here at home. That being said, let's see if anybody upstream sees the same bug on a configuration similar to mine or, failing that, if anybody else using Homebrew does and sees it (should probably have tried the latter first, but oh, well…)
@MikeMcQuaid: You were right, they did ask me to build Qt from source outside of Homebrew (in this comment here.) If that fails, I'll try setting HOMEBREW_MAKE_JOBS=1
like @ilovezfs recommended and see what happens since I haven't gotten around to doing that yet.
As mentioned in this comment on the upstream bug, I'm currently attempting to build Qt from source outside of Homebrew as requested by this comment there. I have, however, run into a snag while trying to do so; specifically, I can't seem to reproduce the exact options that Homebrew passes to Qt's ./configure
script verbatim just from Homebrew's Qt formula. This is not only an issue because I might need to reproduce the Homebrew build environment as closely as possible without invoking it in order to reproduce this issue outside of it, but also because the options I tried choked the configure
stage, causing it to crash with a complaint that it couldn't find MySQL. The run that failed started with an invocation of ./configure -verbose -prefix "/usr/local/Cellar/qt5/5.7.1" -release -opensource -confirm-license -system-zlib -qt-libpng -qt-libjpeg -qt-freetype -qt-pcre -nomake tests -pkg-config -plugin-sql-mysql -I/usr/local/opt/dbus/li/dbus-1.0/include -I/usr/local/opt/dbus/include/dbus-1.0 -L/usr/local/opt/dbus/lib -ldbus-1 -dbus-linked
, which is what, as I currently understand it, is what brew install -vd --build-from-source qt5 --with-dbus --with-docs --with-examples --with-mysql --with-postgresql --with-qtwebkit
or its brew reinstall
equivalent would pass Qt's build scripts after collecting all of them from Homebrew's formula for the software. I also made the same modification to $(QT_BUILD_STAGING_AREA)/qtbase/configure
that the formula does, so what could I be missing (besides -plugin-sql-psql
, which I just realized I forgot, silly me?) Might configure
's behavior depend on its passed-in arguments' order to some extent…? I'm not sure if I copied everything over in the same sequence embodied by the Qt formula's args
container at run-time…
Never mind, I figured it out: turns out all I had to do was temporarily insert raise args.each_with_index {|val, index| puts "args[#{index}] = #{val.inspect}" }
on a new line in between the if build.with? "qtwebkit"…end
block and system "./configure", args
. Now to actually use that information to build Qt outside of Homebrew…
Drat, got the same error message again, but this time I think I know the cause: my involvement in #8714. Now I feel silly…😆.
Oh, wait, no; that's SQLite…facepalm.
OK, my logs for the latest build attempt, this time outside of Homebrew as requested upstream, are here. Like I said before, I can't seem to get past ./configure
even though I was able to inside of Homebrew. Is Homebrew perhaps doing something, maybe with superenv
, that I'm not taking into account here? Any ideas, @ilovezfs or @MikeMcQuaid? If not, maybe Alexandru Croitor from upstream can help like I asked him on the upstream Qt issue in this comment on it…
@RandomDSdevel This could also be related to icu4c, I think I've seen a part of Qt use it in the past. I'd suggest you could narrow things down by trying to build different combinations of options until you can identify which one is the problem.
@ilovezfs and/or @MikeMcQuaid: Here's a thought: am I maybe supposed to set some environment variables before running Qt's configure script that I'm not aware of and am not setting now? Qt's upstream build instructions don't mention/specify any, so I suppose one would assume that said configure script should be able to find everything without help, but here's what I get when put a raise
statement into qt5.rb
just before the line that reads system "./configure", *args
, type '5' to drop into the shell when I get the resulting error, and run printenv
:
HOMEBREW_PROCESSOR=Intel
HOMEBREW_DEV_CMD_RUN=1
TERM_PROGRAM=Apple_Terminal
GEM_OLD_HOME=
HOMEBREW_USER_AGENT=Homebrew/1.1.7 (Macintosh; Intel macOS 10.11.6)
ac_cv_func_basename_r=no
HOMEBREW_LIBRARY=/usr/local/Homebrew/Library
HOMEBREW_COMMAND_DEPTH=1
HOMEBREW_TEMP=/tmp
OBJCXX=clang++
OBJC=clang
SHELL=/bin/bash
TERM=xterm-256color
ac_cv_func_dirname_r=no
MAKEFLAGS=-j2
HOMEBREW_OS_VERSION=macOS 10.11.6
HOMEBREW_CCCFG=sa
TMPDIR=/tmp
HOMEBREW_REPOSITORY=/usr/local/Homebrew
HOMEBREW_ERROR_PIPE=/tmp/homebrew20170117-19450-11sd11u/socket
Apple_PubSub_Socket_Render=/private/tmp/com.apple.launchd.JzhfNAVLwc/Render
TERM_PROGRAM_VERSION=361.1
ac_cv_func_mkostemps=no
SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk
CMAKE_LIBRARY_PATH=/System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries
TERM_SESSION_ID=5C79D555-3A84-4402-9BAA-685BEC90EBA8
HOMEBREW_GIT_CONFIG_FILE=/usr/local/Homebrew/.git/config
HOMEBREW_RUBY_PATH=/System/Library/Frameworks/Ruby.framework/Versions/Current/usr/bin/ruby
HOMEBREW_CXX=clang++
OPAMROOT=/opt/opam
USER=zadmin
TEMP=/tmp
HOMEBREW_OPTFLAGS=-march=core2
SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.ahiV60GFaR/Listeners
HOMEBREW_OPT=/usr/local/opt
HOMEBREW_NO_AUTO_UPDATE=1
__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
HOMEBREW_ANALYTICS_USER_UUID=C5BF68BD-EDC7-499D-8359-CBC029A6BF98
HOMEBREW_CACHE=/Users/zadmin/Library/Caches/Homebrew
ac_cv_func_getentropy=no
ac_have_clock_syscall=no
HOMEBREW_USER_AGENT_CURL=Homebrew/1.1.7 (Macintosh; Intel macOS 10.11.6) curl/7.43.0
PATH=/usr/local/Homebrew/Library/Homebrew/shims/super:/usr/local/opt/dbus/bin:/usr/local/opt/pkg-config/bin:/usr/local/opt/mysql/bin:/usr/local/opt/postgresql/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
HOMEBREW_ISYSTEM_PATHS=/usr/local/include:/usr/include/libxml2:/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers
HOMEBREW_EDITOR=nano
ac_cv_func_clock_gettime=no
PWD=/private/tmp/qt5-20170117-19753-1y2xrxl/qt-everywhere-opensource-src-5.7.1
HOMEBREW_BREW_FILE=/usr/local/bin/brew
ac_cv_search_clock_gettime=no
HOMEBREW_GITHUB_API_TOKEN=1979be4a7b89c59437bae495175ec3dd3c9deca6
HOMEBREW_MACOS_VERSION=10.11.6
PKG_CONFIG_LIBDIR=/usr/lib/pkgconfig:/usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/10.11
LANG=en_US.UTF-8
HOMEBREW_ANALYTICS_ID=UA-76679469-1
HOMEBREW_ARCHFLAGS=
XPC_FLAGS=0x0
ac_cv_func_mkostemp=no
HOMEBREW_PRODUCT=Homebrew
CXX=clang++
XPC_SERVICE_NAME=0
CURL_HOME=/Users/zadmin
SHLVL=2
HOME=/private/tmp/qt5-20170117-19753-1y2xrxl/qt-everywhere-opensource-src-5.7.1/.brew_home
GEM_OLD_PATH=
HOMEBREW_PREFIX=/usr/local
CMAKE_INCLUDE_PATH=/usr/include/libxml2:/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers
TMP=/tmp
MACOSX_DEPLOYMENT_TARGET=10.11
LOGNAME=zadmin
HOMEBREW_LIBRARY_PATHS=/usr/local/lib:/System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries
HOMEBREW_PREFER_CLT_PROXIES=1
HOMEBREW_CC=clang
PKG_CONFIG_PATH=/usr/local/opt/dbus/lib/pkgconfig:/usr/local/opt/mysql/lib/pkgconfig:/usr/local/opt/postgresql/lib/pkgconfig
HOMEBREW_CASK_OPTS=--appdir=/Applications --prefpanedir=/Library/PreferencePanes --qlplugindir=/Library/QuickLook --fontdir=/Library/Fonts --input_methoddir=/Library/Input\ Methods --screen_saverdir=/Library/Screen\ Savers
HOMEBREW_OPTIMIZATION_LEVEL=Os
HOMEBREW_VERSION=1.1.7
ACLOCAL_PATH=/usr/local/share/aclocal
HOMEBREW_CELLAR=/usr/local/Cellar
ac_cv_func_timingsafe_bcmp=no
CMAKE_PREFIX_PATH=/usr/local
CC=clang
DISPLAY=/private/tmp/com.apple.launchd.6VT1gsjF0u/org.macosforge.xquartz:0
HOMEBREW_CURL=/usr/bin/curl
HOMEBREW_DEPENDENCIES=dbus,pkg-config,mysql,postgresql
ac_cv_func_clock_settime=no
HOMEBREW_FORMULA_PREFIX=/usr/local/Cellar/qt5/5.7.1_1
ac_cv_func_clock_getres=no
HOMEBREW_SYSTEM=Macintosh
_=/usr/bin/printenv
Of these environment variables, the following jump out at me as possibly affecting how Homebrew builds Qt when running the related formula:
OBJCXX
OBJC
MAKEFLAGS
SDKROOT
CMAKE_LIBRARY_PATH
HOMEBREW_CXX
HOMEBREW_ISYSTEM_PATHS
PKG_CONFIG_LIBDIR
CXX
CMAKE_INCLUDE_PATH
HOMEBREW_LIBRARY_PATHS
HOMEBREW_CC
PKG_CONFIG_PATH
ACLOCAL_PATH
CMAKE_PREFIX_PATH
CC
Of these, it looks like the values in the HOMEBREW_*
variables might get passed either through to their non-HOMEBREW_*
equivalents or through directly to Qt's configure script as options. HOMEBREW_DEV_CMD_RUN
was most likely set to 1 by Homebrew itself while setting itself up, as it usually isn't on a routine basis. HOMEBREW_NO_AUTO_UPDATE
was, however, set by me, as I tend to prefer more explicitness in my operations. I'm a little confused why I'm seeing CMake-related things pop up here, though, as, IIRC, Qt uses qmake
internally after its build is triggered by make
, though Qt does generate CMake bindings, so maybe that's what those variables are for…? In any case, I don't think its any of the Qt formula's options that are causing me issues unless they muck with QtBase, QtCore, QtWebEngine, or one of that last module's other dependencies somehow, since the failures I keep seeing inside Homebrew always happen when the build gets to linking QtWebEngine's Demobrowser application's internal object file. It's getting past this MySQL-misdetection–related glitch that only appears outside of Homebrew using stuff from inside of it that's giving me trouble, as you can see by my comments on the issues I've filed both here and upstream. For what it's worth, at least I know I'm doing something wrong; I just can't figure out what it is…sigh.
@ilovezfs and/or @MikeMcQuaid: I'd just like to note that I recently updated my previous comment on this thread with a few more details, not to mention '@'-mentions of your usernames.
On second thought, perhaps HOMEBREW_DEV_CMD_RUN
got set by me (even though I don't remember seeing it in the results of printenv
beforehand) when I ran brew edit
(I can't remember off the top of my head if that's a dev-cmd or not…goes to man brew
to check… — yes, I guess it is; huh, go figure…/facepalms again) to printf
-debug "$(brew --repository)/Formula/qt5.rb"
. In any case, a quick run of the following commands…:
cd "$(brew --repository)"
git config --local --replace-all --bool homebrew.devcmdrun false
git checkout -B stable 1.1.7
…quickly fixed that bit of confusion, as it looked likebrew
's Git homebrew.devcmdrun
configuration flag had gotten set as well. (That reminds me that maybe I should go suggest an addition to brew update-reset
that does this for end users, albeit without them having to specify the latest release tag manually…)
@RandomDSdevel Please try to narrow down which option(s) are failing, thanks.
@MikeMcQuaid:
I thought I said something to that effect, but perhaps it got buried in the middle of one of my (admittedly rather large) info dumps, in which case I'll repeat myself: as far as I can tell from what I've seen so far, no options are failing when I build Qt inside Homebrew — unless I'm overlooking an optional dependency, of course —, but the build process still fails to link QtWebEngine's Demobrowser application bundle's internal executable. When trying to reproduce this failure outside of Homebrew as requested upstream using as close of an approximation to the environment that it sets up for a Qt build as I can, I can't even get that far, as the configure stage fails to find MySQL for who knows what reason, though I suspect it has something to do with something Homebrew does prior to running configure
that I haven't thought of, which is why I'm asking for help down here rather than upstream for now. Given that the reason that I have to build Qt outside of Homebrew in the first place is that make
's error report isn't verbose enough, however, I might just see if there's something I could add to qt5.rb
's make
invocation to increase make
's verbosity to a more informative level (but not too overwhelming of one,) if I can't eventually manage to build Qt outside of Homebrew.
@MikeMcQuaid: I have been looking at open brew
executable GitHub issues for a bit on a whim unrelated to this issue and just saw Homebrew/brew#1034 while doing so. Could that be why the output I'm seeing from make
when building Qt using/inside Homebrew is somewhat muted? It would be nice if I could get the error output needed to pinpoint the root cause of this issue without having to mess around with trying to build Qt outside of Homebrew since that hasn't been working for me for whatever reason…
@RandomDSdevel It certainly looks like the description of https://github.com/Homebrew/brew/issues/1034 might be the issue of why you are not seeing make's error output when building with brew.
Regarding compiling qt outside of brew, please try to remove all configure options regarding sql databases, and see if you can get to the same point that QtWebEngine's demobrowser fails.
By looking at https://github.com/RandomDSdevel/Homebrew-Installation-Logs/blob/master/Qt/v5.7.1/Third%20Attempt/qt5-configure.out , it looks like the configure test for the mysql plugin can't find the openssl library, and that basically fails the whole configure process because specifying the mysql plugin is a hard requirement in this case. It actually says so plainly at the end of the log:
ld: library not found for -lssl
MySQL (thread-unsafe) disabled.
MySQL support cannot be enabled due to functionality tests!
The fact that you change something inside of configure (as the qt5 formula) regarding ssl and crypto libraries, might be why you are seeing errors when compiling outside of brew.
@Placinta:
@RandomDSdevel It certainly looks like the description of Homebrew/brew#1034 might be the issue of why you are not seeing make's error output when building with brew.
⋮
I'll continue to keep an eye on how that issue develops, then.
⋮
Regarding compiling qt outside of brew, please try to remove all configure options regarding sql databases, and see if you can get to the same point that QtWebEngine's demobrowser fails.
Yeah, that's probably my next step. I've been trying to minimize the number of Qt builds I have to do to get enough error output to resolve this issue, though, as, like I've mentioned before in various places, the computer I'm using is so old that I typically have to fully-featured Qt builds run overnight (I'm working on getting a newer machine, but it's taking a lot longer than I would like.)
⋮
By looking at https://github.com/RandomDSdevel/Homebrew-Installation-Logs/blob/master/Qt/v5.7.1/Third%20Attempt/qt5-configure.out , it looks like the configure test for the mysql plugin can't find the openssl library, and that basically fails the whole configure process because specifying the mysql plugin is a hard requirement in this case. It actually says so plainly at the end of the log:
ld: library not found for -lssl MySQL (thread-unsafe) disabled. MySQL support cannot be enabled due to functionality tests!
⋮
That's what's got me puzzled, as this error doesn't occur when building inside of Homebrew.
@Placinta:
The fact that you change something inside of configure (as the qt5 formula) regarding ssl and crypto libraries, might be why you are seeing errors when compiling outside of brew.
I made that particular change because Homebrew does, too, on lines 75–78 of (facepalm: you already knew that; I just failed to parse that you did…) "$(brew --repository homebrew/homebrew-core)/Formula/qt5.rb
. Strangely enough, however, the relevant inreplace
call looks like it could be a no-op given Qt's source…
Since Homebrew makes the exact same change, I don't think that's the problem.
OK, successfully ./configure
d outside of Homebrew without passing said script its -plugin-sql-mysql
or -plugin-sql-psql
options. Now let's try make
…
OK, make
failed this time, too, but with yet another different error, and not nearly as far into the build process as I expected! I'll run the exact same sequence of steps tomorrow with redirection of stdout
and stderr
to one file each for configure
and make
and post a link to the results here in this issue thread. I'm starting to think that I might want to wait until Homebrew/brew#1034 gets resolved to get debug output by running my original brew reinstall qt5 …
command again, though, as this is starting to get rather annoying…
Hold on, now I feel stupid: I forgot to make sure my build environment was clean and didn't contain anything left over from previous build attempts. Next time, I'll remember, as I've decided to start from scratch then.
@ilovezfs, @MikeMcQuaid, and/or @Placinta: OK, here are the logs that resulted from me following through on the promise I made in this comment above. I'll note that something weird happens with Freetype during ./configure
, which may have something to do with the error that make
spews out.
I'll explain what happens with configure.
Your configure line specifies the usage of the bundled qt version of FreeType. But you also specify the use of pkg-config, and because configure by default checks if fontconfig is available (because pkg-config is set), it resorts to use system freetype (that's a requirement of using fontconfig).
Now, from a little research I assume the following things: 1) fontconfig is not supported on mac (see https://bugreports.qt.io/browse/QTBUG-42839 ) 2) Qt bundled freetype is supposed to be used for macOS (see discussion and commit in https://github.com/Homebrew/homebrew-core/issues/188 )
So I would advise you either not to pass -pkg-config, or to pass -no-fontconfig .
@RandomDSdevel note that we've just shipped Qt 5.8.0 so you'll probably want to go back to square one now and see if there's still any problem(s) for you.
@Placinta: Huh, I swear that's what Homebrew did internally. Guess I must have misread the configuration options I extracted from it earlier or something…?
@ilovezfs: OK, thanks! Hopefully that does the trick…
@ilovezfs: Hurrah, Qt 5.8.0 built and installed successfully under Homebrew as recorded here! Still, I do wonder what exactly was going wrong during my other, failed build attempts…
@RandomDSdevel you're in luck. You can continue attempting to build qt@5.7 now also: https://github.com/Homebrew/homebrew-core/pull/9297
@ilovezfs: Meh, not sure if that would be worth the trouble. I might try it one more time after Homebrew/brew#1034 gets resolved so I don't have to unsuccessfully try jumping through all the hoops necessary to build Qt outside of Homebrew in an environment resembling the one it uses to build the package inside itself as closely as possible again just to get decent logs, but, right now, I'm just happy I have the latest version of Qt built and installed via Homebrew.
@RandomDSdevel Note that there are formulae depending on qt@5.7 so you may "have to" build it anyway :)
codequery
gammaray
gnuplot (optional)
gpsbabel
pyqt5
qscintilla2
qwt
qwtpolar
homebrew/games/pc6001vx
homebrew/science/octave
homebrew/science/vtk (optional)
@ilovezfs: Nope, not using any of those yet. Thanks for the heads-up, though.
Please follow the general troubleshooting steps first:
brew update
and retried your prior step?brew doctor
, fixed as many issues as possible and retried your prior step?Bug reports:
Well, now I feel even more stupid than I did before. After reading the advice that @MikeMcQuaid left via this comment in the mess that was this issue just prior to resolving it, I apparently forgot about it and went ahead and manually mucked about with Homebrew state by running
brew uninstall -vd --force --ignore-dependencies qt qt5
or some such insanity on my part… That being said, however, I'm not entirely sure that's really the point here. After all, I'm pretty sure I would have seen the same problem crop up had I just waited for 1.1.6 to be released and bring the relevant fix for my previous craziness to my localstable
branch. Anyway, what you probably want to hear is the actual bug report, so I'd better write that out in the next paragraph once I'm done miming bashing my head into a wall in this one… What happened was this: attempting to install Qt v5.7.1--with-dbus
,--with-docs
,--with-examples
,--with-mysql
,--with-postgresql
, and--with-qtwebkit
failed under abrew config
similar to this one — my apologies for not logging what I had prior to runningbrew update
today. My logs are here. Please note, however, that, per this issue that I opened on the logs repository I created when I got tired of scrolling through the lengthy output ofbrew gist-logs
on GitHub Gists (which you've probably seen links to before in some of my issues,) I had to compress some of them usinggzip
in order to get them under GitHub's 100-MB file-size limit. I haven't skimmed the end of the logs yet, but, if I do manage to figure out what went wrong on my now without any help, I'll file a pull request to fix it. In the meantime, the more eyeballs, the better. (When will the problems with building Qt ever end? I swear that's one of the few packages I have trouble building from source… — yes, I know that's not recommended…)