IBM / CAST

CAST can enhance the system management of cluster-wide resources. It consists of the open source tools: cluster system management (CSM) and burst buffer.
Eclipse Public License 1.0
27 stars 34 forks source link

How to build a new copy of BBAPI using a newer version of BOOST #1020

Open roymuss opened 2 years ago

roymuss commented 2 years ago

Question

We have started integrating the SCR library into our code base. However, our code requires BOOST version 1.74.0 to compile (header-only). When we use SCR with BBAPI, it hangs during one of the boost calls. I believe that BBAPI is built with 1.53.0 and this is incompatible with our version. Is there any possibility of an installation of BBAPI using the newer version of Boost? Or have an implementation that is more robust to conflicting boost versions?

I do see some install steps in docs in the github repo. However, it'd be good to get IBM's suggestion and help on the next steps. -Roy Musselman roymuss@us.ib.com

Answer:

_place

Approach:

here replace this with a short summary of how you addressed the problem. in the comments place step by step notes of progress as you go

What is next:

Define the next steps and follow up here

tgooding commented 2 years ago

BBAPI uses RHEL8's system boost library, which is 1.66.0-10. The old releases that were based on RHEL7 used boost 1.60.0-4 (for boost::log).

The analysis presented here does not seem to be complete as header-only usage w/o linking 1.74 boost libraries shouldn't ripple outside of the SCR object file. There is more to that story (memory leak/corruption? something unexpected happening with dynloader (use LD_DEBUG))

That said, the side-channel discussion said the hang appeared within a C++ static constructor. We agree thats not a good hygiene, commit #1021 makes a change to connections.cc to avoid a C++ static constructor.

To build, the directions are generally:

% git commit https://github.com/IBM/CAST.git
% cd CAST

We use a build container to build CAST, you should be able to create the build container via:  
%  podman build --rm -t castbuild:8.4 -f scripts/Dockerfile

Then build via:
%  podman run --rm -v $PWD:/home/build castbuild:8.4 scripts/configure.pl --rpmbuild --skipcsm
%  podman run --rm -v $PWD:/home/build castbuild:8.4 scripts/rebuild noinstall package

This will create RPMs in work/rpms.

roymuss commented 2 years ago

Hi Tom, I forwarded your instructions to Adam Moody and he had the following reply. (Note the LLNL systems are still at RHEL 7.6)

Thanks, Roy. We could give that a shot to see if it helps.

Would you please ask IBM if they could forward a fresh rpm build to us?

Is that something that we could just unpack as a side install for testing rather than replacing the rpm on the system?

I'll also run these steps by Dave Fox to see if we can build the BBAPI library in house. I think being able to do that could be helpful in the future. -Adam


Roy Musselman
IBM HPC Application Analyst at Lawrence Livermore National Lab
email: ***@***.***
LLNL office: 925-422-6033
Cell: 507-358-8895, Home: 507-281-9565

From:   "Tom Gooding" ***@***.***>
To: "IBM/CAST" ***@***.***>
Cc: "roymuss" ***@***.***>, "Author"
            ***@***.***>
Date:   02/22/2022 10:40 AM
Subject:    [EXTERNAL] Re: [IBM/CAST] How to build a new copy of BBAPI
            using a newer version of BOOST (Issue #1020)

BBAPI uses RHEL8's system boost library, which is 1.66.0-10. The old
releases that were based on RHEL7 used boost 1.60.0-4 (for boost::log). The
analysis presented here does not seem to be complete as header-only usage
w/o linking 1.74 boost ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
ZjQcmQRYFpfptBannerEnd

BBAPI uses RHEL8's system boost library, which is 1.66.0-10. The old
releases that were based on RHEL7 used boost 1.60.0-4 (for boost::log).

The analysis presented here does not seem to be complete as header-only
usage w/o linking 1.74 boost libraries shouldn't ripple outside of the SCR
object file. There is more to that story (memory leak/corruption? something
unexpected happening with dynloader (use LD_DEBUG))

That said, the side-channel discussion said the hang appeared within a C++
static constructor. We agree thats not a good hygiene, commit #1021 makes a
change to connections.cc to avoid a C++ static constructor.

To build, the directions are generally:

% git commit https://github.com/IBM/CAST.git
% cd CAST

We use a build container to build CAST, you should be able to create the
build container via:
%  podman build --rm -t castbuild:8.4 -f scripts/Dockerfile

Then build via:
%  podman run --rm -v $PWD:/home/build castbuild:8.4 scripts/configure.pl
--rpmbuild --skipcsm
%  podman run --rm -v $PWD:/home/build castbuild:8.4 scripts/rebuild
noinstall package

This will create RPMs in work/rpms.

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you authored the thread.Message ID:
***@***.***>