[Formula/open-mpi] Open MPI Fails to Build and Install Under OS X v10.11.6 'El Capitan'

[RandomDSdevel]

Please note we will close your issue without comment if you delete, do not read, or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.

Please always confirm that you:

• [x] are reporting a bug others will be able to reproduce and not asking a question. If you're not sure or want to ask a question, do so on our Discourse: https://discourse.brew.sh.
• [x] have a problem with brew installing (or upgradeing or reinstalling) a single, official formula (not a cask?) If it's a general brew problem, please file this issue at Homebrew/brew: https://github.com/Homebrew/brew/issues/new. If it's a brew cask problem please file this issue at https://github.com/caskroom/homebrew-cask/issues/new. If it's a tap — e.g. Homebrew/homebrew-php — problem, please file this issue at the tap.
• [x] ran brew update and can still reproduce the problem? (Caveat: I suspended running brew updates while producing debugging logs for this issue to ensure that Homebrew Core would stay on the same revision between both log-collection runs I made, so me checking this box only applies with respect to the status of my system prior to my preparing to submit this issue. This should not, however, prove to be any sort of problem, as no commits to Homebrew Core since I started going down this rabbit hole appear to have touched any files that would be relevant here.)
• [x] ran brew doctor, fixed all issues and can still reproduce the problem?
• [ ] ran brew gist-logs <formula> (where <formula> is the name of the formula that failed) and included the output link? (N/A; see the next bullet point.)
• [x] if brew gist-logs didn't work (or, in this case, was not preferred:) ran brew config and brew doctor and included their output with your issue?

To help us debug your issue please explain:

• What you were trying to do (and why:)

       Upgrade my local, built-from-source copy of Open MPI from v3.0.0, Homebrew formula revision 2, to v3.0.1 in the course of going through my normal routine of keeping all of my software tools up to date.

• What happened (include command output:)

       Homebrew failed to successfully build Open MPI from v3.0.0, Homebrew formula revision 2, to v3.0.1. My logs are:

  • here (from when I did set HOMEBREW_BUILD_FROM_SOURCE=1.)
  • here (from when I didn't.)

  In both cases, the exact point of failure manifested itself as follows:

  ⋮
  Making all in mpi/java
  Making all in java
   JAVAC    MPI.class
  Picked up _JAVA_OPTIONS:  -Duser.home=/Users/zadmin/Library/Caches/Homebrew/java_cache
   JAVAH    mpi_MPI.h
   JAVAH    mpi_CartParms.h
  Unable to locate an executable at "/Library/Java/JavaVirtualMachines/jdk-10.jdk/Contents/Home/bin/javah" (-1)
  Unable to locate an executable at "/Library/Java/JavaVirtualMachines/jdk-10.jdk/Contents/Home/bin/javah" (-1)
  make[3]: *** [mpi_CartParms.h] Error 2
  make[3]: *** Waiting for unfinished jobs....
  make[3]: *** [mpi_MPI.h] Error 2
  make[2]: *** [all-recursive] Error 1
  make[1]: *** [all-recursive] Error 1
  make: *** [all-recursive] Error 1
  /usr/local/Homebrew/Library/Homebrew/build.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/python.rb
  /usr/local/Homebrew/Library/Homebrew/debrew.rb:11:in `raise'
  BuildError: Failed executing: make all
  1. raise
  2. ignore
  3. backtrace
  4. irb
  5. shell
  Choose an action: 1
  ⋮

  The issue I describe in this report is thus due to Homebrew's open-mpi formula's --with-java option and the fact that javah was deprecated in Java 9.x and removed in Java 10.

• What you expected to happen:

       For Homebrew to successfully build the new version of Open MPI from source and install it in the old version's place.

• Step-by-step reproduction instructions (by running brew commands:)

  • With HOMEBREW_BUILD_FROM_SOURCE=1 (not supported with issue resolution:)

    HOMEBREW_BUILD_FROM_SOURCE=1 brew upgrade -vd --build-from-source open-mpi --with-java --with-mpi-thread-multiple

    Note that setting that environment variable as shown above probably doesn't make any difference in this case, as upgrading Open MPI doesn't depend on upgrading any of its dependencies given the states my Homebrew installation and the packages it manages are currently in, but I usually set it any time I run brew upgrade, just to be consistent.

  • Without HOMEBREW_BUILD_FROM_SOURCE=1 (given to show that this issue also occurs under a supported configuration:)

    brew upgrade -vd --build-from-source open-mpi --with-java --with-mpi-thread-multiple

[scpeters]

I see open-mli in the last two commands; I'm guessing that's a typo and it should be open-mpi?

[RandomDSdevel]

@scpeters: Oof, should have looked for more of those; I saw one, but didn't catch the ones you saw before submitting this issue. I'm fixing them now…grumbles something incoherent about either auto-correct, having intermittent butterfingers, or both.)

[RandomDSdevel]

@scpeters: Reported upstream as open-mpi/ompi#5000.

[RandomDSdevel]

You know, I'm surprised Open MPI's configure.ac script didn't detect this and bail because of it…

[RandomDSdevel]

     Looking through my logs, it appears said script fell back to using some programs it found in /ur/bin, which…doesn't make sense, as the JDK installs into /Library/Java/JavaVirtualMachines, and I don't think I'd have detritus from any of the old Apple-vended Java installations left lying around (as far as I can remember, this OS X installation hails from after Apple handed releasing Java for Macs back over to Oracle.) I wonder where those executables came from…?

[RandomDSdevel]

     Never mind; I don't see any old Apple Java run-time libraries left over on my machine in /System/Library/Java (there's no JavaVirtualMachines folder in there.) What I'm dealing with here must therefore all be from within /System/Library/Frameworks/JavaVM.framework, which is apparently some infrastructure that's still needed even with the Oracle JREs and JDKs. (Per an old message which I found on one of the OpenJDK mailing lists via this Ask Different answer when I was looking around to see what information I could jump off from to troubleshoot and make sure I was as free of old Apple Java installations as I thought I was:

⋮

Do NOT remove any content in the JavaVM.framework. Those items are required by Java 7, 8, 9+ as well as Java SE 6. No modern version of OS X has a Java JDK inside there anyway.

⋮

So, in any case, I'm not crazy and did end up cleaning up after any old Apple Java VMs I might ever have had, if any.)
     Anyway, if I poke at /usr/bin/javah (or /System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands/javah, the former's symbolic-linking target which behaves identically) — say, by invoking it with its --version or --help flag —, then here's what I get back:

Unable to locate an executable at "/Library/Java/JavaVirtualMachines/jdk-10.jdk/Contents/Home/bin/javah" (-1)

If I ask my shell what $? is after that, then it prints '2,' an error code. This, of course, is all to be expected, given the circumstances.

[RandomDSdevel]

@scpeters:

     It's getting a bit late, so I'll probably sign off for the night soon, but I may have some idea of what would need to be patched upstream to get this to work, so I'll hack around on that tomorrow. (Or we could just make sure that Apple's shims don't come first in users' paths ourselves down here in Homebrew, but that feels like a hack, as it would make more sense for upstream to know not to look in /usr/bin for JDK executables on OS X/macOS in the first place…)

[RandomDSdevel]

     I've now brought this issue's upstream counterpart to the attention of the Open MPI developers' mailing list here and have already received a response to the thread that my former message started there. This reply was to the effect that one of Open MPI's developers felt this issue to be important enough to submit at least a partial fix for it himself. I'll still be working on finishing things up, though (OS X/macOS JDK tool path handling still needs correcting,) so that's till coming.

[RandomDSdevel]

     This issue will be addressed upstream once open-mpi/ompi#5001 and open-mpi/ompi#5015 get merged. I'll submit a formula patch to pull them in once they are (I doubt those two changes alone would be enough for upstream to tag a new release…)

[RandomDSdevel]

     #5001 has now been merged upstream to master as commit 5e12e0f and is currently in the process of being backported to Open MPI v3.0.x in open-mpi/ompi#5029 pending upstream CI network hiccups. open-mpi/ompi#5015 should follow shortly.

[fxcoudert]

Thanks @RandomDSdevel for chasing the bug and reporting it. Given that it affects a non-default option, I think we'll wait for the fix in the next version.

[RandomDSdevel]

@fxcoudert:

     I was going to submit the relevant patch in-formula to fix this once said commit landed upstream since I'm the one who's working on that anyway, but OK.

[RandomDSdevel]

     Not that this is nearly as relevant before since @fxcoudert has indicated that Homebrew Core is just going to wait for the Open MPI v3.0.2 to get this fix and the previous one relevant here and has not indicated interest in reopening this issue for me to submit a formula patch to incorporate both of them, but open-mpi/ompi#5015 has now been merged to master upstream.

[RandomDSdevel]

     I'm going to go ahead and follow up on this issue by submitting a PR to fix it even though it was never reopened for this to happen.