search

plfs / plfs-core

LANL no longer develops PLFS. Feel free to fork and develop as you wish.
41 stars 36 forks source link

Patching MPI to not require LD_LIBRARY_PATH to be set... #320

Closed thewacokid closed 10 years ago

thewacokid commented 10 years ago

Find a way to patch MPI with PLFS so that users don't need to have PLFS in their LD_LIBRARY_PATH

thewacokid commented 10 years ago

Okay so here's the status on this:

Moving back to the old patching/building steps gets rpath/runpath set in the OpenMPI library. This adds the offending "-lplfs" lines back into the mpicc wrapper which means everyone using a custom PLFS must point the compiler/linker at the correct version of PLFS [i]and[/i] setup LD_LIBRARY_PATH to point to the correct version of PLFS for their application (otherwise system versions will be found at runtime regardless of what was used to build OpenMPI).

So, instead of requiring LD_LIBRARY_PATH to be set at runtime we require linking to PLFS (assuming the correct version is not in the system path) through mpicc and again require LD_LIBRARY_PATH at runtime (same assumption as before).

So the trade ends up being that most people wouldn't have to deal with the PLFS libs at all with a PLFS-aware OpenMPI as long as they aren't trying to use PLFS directives in their code. Those that do want to use PLFS routines in their code wouldn't have to change anything if the system version of PLFS matches the system version of MPI.

Those that want to use a custom version of PLFS will have to jump through hoops.

I would guess that is acceptable?

dshrader commented 10 years ago

By the old method do you mean setting LDFLAGS, CFLAGS, etc. for the top-level configure line (through a platform file or env variables)?

thewacokid commented 10 years ago

Exactly. But there was a problem with that now that I think about it...didn't an old PLFS install mess with things?

dshrader commented 10 years ago

I don't remember the actual situation we ran in to that prompted us to change... Did it have something to do with -pthread?

thewacokid commented 10 years ago

Ah, yes. Wasn't it that Autotools was picking up the old deps from the system install and making the new build do weird things? I don't recall how I replicated the build failures related to the pthreads lib...

atorrez commented 10 years ago

Yes pthread error when building.

From: David notifications@github.com<mailto:notifications@github.com> Reply-To: plfs/plfs-core reply@reply.github.com<mailto:reply@reply.github.com> Date: Wed, 23 Oct 2013 09:40:30 -0700 To: plfs/plfs-core plfs-core@noreply.github.com<mailto:plfs-core@noreply.github.com> Subject: Re: [plfs-core] Patching MPI to not require LD_LIBRARY_PATH to be set... (#320)

I don't remember the actual situation we ran in to that prompted us to change... Did it have something to do with -pthread?

— Reply to this email directly or view it on GitHubhttps://github.com/plfs/plfs-core/issues/320#issuecomment-26921273.

thewacokid commented 10 years ago

Okay so with a clean environment I can build/install PLFS and OpenMPI through the regression suite. RPATH and RUNPATH are both set to the proper paths for libmpi.so and associated libraries (verified with objdump -p).

I'll finish up my testing to make sure things are working as I think they are, update the documentation, then submit pull requests for both the regression suite and plfs.

thewacokid commented 10 years ago

Question time...

In the past we've used "-Wl,--whole-archive -lplfs -lpthread -Wl,--no-whole-archive" in our build scripts (but not in our patch notes). Why are we using --whole-archive for the regression suite for lplfs and lpthread?

dshrader commented 10 years ago

If I have all of this figured out, we use those flags for static builds/libraries. That way the linker pulls in all of libplfs in to the static library instead of just the pieces it needs. I don't think we need pthreads inside the whole-archive thing, though.

thewacokid commented 10 years ago

Do we want MPI to pull in the whole library of anything? Even for static builds I would think we would just want the routines we use and not the entirety of the library.

dshrader commented 10 years ago

I don't really know. The only case that I would worry about is someone doing PLFS calls in their own code, but then they should be linking in PLFS explicitly. So maybe we don't need to pull everything in for a patched MPI...

thewacokid commented 10 years ago

Exactly my thoughts - if someone is using PLFS explicitly they should be linking to PLFS themselves, not relying on the MPI lib to have the proper definitions.

thewacokid commented 10 years ago

Reverted changes from https://github.com/plfs/plfs-core/issues/312