search

optimizers / homebrew-fenics

Building FEniCS on OSX with Homebrew and Pip.
GNU General Public License v3.0
3 stars 4 forks source link

Isolating PETSc downloaded libraries #3

Closed goxberry closed 8 years ago

goxberry commented 10 years ago

Something I noticed with the installation process is that the PETSc libraries will be symlinked into $(brew --prefix)/lib and the PETSc headers will be symlinked into $(brew --prefix)/include. Some of these headers and libraries will conflict with installed headers and libraries from other packages, like:

In the short term, brew link --overwrite petsc-fenics is fine; in the longer term, is it possible to isolate the PETSc versions of libraries and headers from system versions of these packages? Some PETSc packages have patches that have not been migrated to upstream, so it's not clear to me that PETSc should use the system packages instead of downloading its own. It's also not clear if there will be later conflicts with other packages that depend on METIS, ParMETIS, etc., or if other software later installed outside of Homebrew will pick up the wrong headers and libraries as a result of the symlink conflicts. The version of PETSc from Homebrew core would probably experience similar conflicts if it downloaded external packages for PETSc.

I'm willing to help write a patch and discuss ideas, if you think this behavior is an issue worth resolving.

dpo commented 10 years ago

My idea was to phase out the petsc-fenics formula in favor of the one in homebrew, as it has dependencies managed by homebrew, instead of by PETSc itself. Luke reported having some trouble with that, so I'd like to debug the process. I suppose that in the mean time, we could instruct this PETSc formula to install in subdirs, or to be keg only. @lukeolson what do you think?

goxberry commented 10 years ago

@dpo: The homebrew-science version of petsc doesn't install any external packages (METIS, etc.) whatsoever, nor does it pull in external packages from existing homebrew-science formula (again, METIS is a good example). For some external packages, I think using the homebrew provided versions will be fine; MacPorts provides some packages for PETSc. The PETSc developers patch some of those external packages to work with PETSc, it's important to sort out what external packages have to be downloaded by PETSc.

My impression of the homebrew-science version of PETSc is that it's exceedingly limited as is, because it downloads no external packages. And due to the symlink issue, naïvely adding optional dependencies won't work well. Whenever PETSc is compiled from scratch into a local directory, many of the relevant libraries are installed to ${PETSC_DIR}/${PETSC_ARCH}/lib and many of the relevant headers are installed to ${PETSC_DIR}/${PETSC_ARCH}/include. Although PETSc's external makefiles (for projects using PETSc libraries) use rules in ${PETSC_DIR}/conf/rules and ${PETSC_DIR}/conf/variables, these makefiles themselves reference ${PETSC_ARCH}. My first hack at isolating the libraries would be to symlink the libraries PETSc downloads into something like $(brew --prefix)/lib/petsc/${PETSC_ARCH} and symlink headers into something like $(brew --prefix)/lib/petsc/${PETSC_ARCH}.

As for petsc-fenics, I guess it all depends. The Dorsal package for PETSc adds a couple compile flags (the important ones added are: --with-clanguage=cxx and --with-c-support=1), so I configured and compiled PETSc with a different value of PETSC_ARCH just for FEniCS when I was attempting to install DOLFIN manually. I could see the utility of having one PETSc install for everything, and I could also see cases where having a separate FEniCS install would be helpful. One example: if you ask for help on the petsc-support mailing list, they'll ask you to compile PETSc with --with-debugging=1; for use with FEniCS, you probably want to use --with-debugging=0 almost all the time, because PETSc is just being used as a backend.

lukeolson commented 10 years ago

I played with this a few weeks ago and just pushed it in to a new branch: https://github.com/optimizers/homebrew-fenics/blob/newpetsc/petsc-fenics.rb

I'm not sure why the packages would need patches (there may be a very good reason). But I was able to get the symlinks working with everything except Hypre (only because there was no formula .... yet). Let me know what you think; this approach would allow stripped down PETSc installs (as currently offered in homebrew-science) but would also allow linking to the installed homebrew packages if they exist. In summary, I'm proposing that this petsc-fenics.rb could replace the standard one.

Three issues remain: 1. Is this best way to handle the links? So far it seems fine. 2. Should an additional option "as-downloads" or something be added to bypass the homebrewed versions. This may be dangerous though as @goxberry suggests in started this thread. And 3. A Hypre formula is needed, but should be easy.

goxberry commented 10 years ago

@lukeolson:

I played with this a few weeks ago and just pushed it in to a new branch: https://github.com/optimizers/homebrew-fenics/blob/newpetsc/petsc-fenics.rb

Looks great! PETSc 3.5 will replace --with-umfpack with --with-suitesparse, just as a heads up.

I'm not sure why the packages would need patches (there may be a very good reason). But I was able to get the symlinks working with everything except Hypre (only because there was no formula .... yet). Let me know what you think; this approach would allow stripped down PETSc installs (as currently offered in homebrew-science) but would also allow linking to the installed homebrew packages if they exist. In summary, I'm proposing that this petsc-fenics.rb could replace the standard one.

So, for example PETSc developers have:

I don't know if PETSc is compatible with METIS 4 and things just work fine, or if it requires METIS 5. I use PETSc's default build system and have PETSc download METIS because that works very well, and they sandbox their libraries, which is generally good until you start building packages that link to PETSc and some other version of those libraries (e.g., FEniCS) because that causes linking issues. (For example Douglas Arnold noticed a linking issue in building FEniCS with a different version of SCOTCH than the version of SCOTCH used by PETSc. See http://fenicsproject.org/qa/634/dorsal-build-of-slepc-fails.) Taking that perspective, it is probably better for Homebrew (and Homebrew-science) to use system installs of packages for the sake of composing different Homebrew packages. Also, using that rationale, the symlinking issue goes away, and it becomes mainly a bug patching issue.

If the idea is to install external packages via Homebrew, maybe the MUMPS patches could be added to the MUMPS formula in Homebrew? I've used them to compile MUMPS by hand, and then used that version of MUMPS in manually compiling IPOPT, and it worked beautifully. I don't know if the Chaco patches will play so nicely with other libraries (aside from PETSc).

The issue of debugging flags might be better addressed in a separate thread; if PETSc external packages are linked from Homebrew, I guess they would need to be built with debugging flags as well? I keep raising it only because, having sought help from these mailing lists before, they generally ask for things that require a debug version.

dpo commented 8 years ago

I'm going to close this as almost everything has changed since this issue was opened. I just merged updates to version 1.6.0 of all packages. Please open a new issue if there are still difficulties.