Removing GCC 4 compiler support

[jessemapel]

Right now, we support compiling ISIS with two versions of GCC, 4 and 7. This has allowed us to continue to support development and compilation on old CentOS systems, but requires some additional work. Binaries compiled with GCC are forwards compatible, but not backwards compatible. That is, libraries compiled with GCC 4 are compatible with libraries compiled with GCC 7, but not visa-versa. To maintain, GCC 4 support we have been serving some of ISIS's third party dependencies compiled with GCC 4, because the binaries available on conda-forge were compiled with GCC 7.

Recently, conda-forge (the main source of our dependencies) changed from using GCC 4 to compile all Linux binaries to using GCC 7. This makes updating to new versions of dependencies and adding dependencies problematic. We either have to serve the new versions compiled with GCC 4 ourselves or keep the CentOS build on old versions. Both options are not ideal. Serving dependencies ourselves defeats the purpose of using conda-forge and adds significant maintenance overhead. Keeping the CentOD build on old versions could require significant changes in the code base and build system.

For these reasons, the best options appears to be dropping support for compilation with GCC 4. ISIS will still run on systems that use GCC 4, because the GCC 7 ISIS binaries can run on older systems, but development will no longer be supported using GCC 4. I would recommend making this change with version 4.0 coming out next month. We can maintain GCC 4 support in 3.9.

[ascbot]

I am a bot that cleans up old issues that do not have activity.

This issue has not received feedback in the last six months. I am going to add the inactive label to this issue. If this is still a pertinent issue, please add a comment or add an emoji to an existing comment.

I will post again in five months with another reminder and will close this issue on it's birthday unless it has some activity.

[ascbot]

I am a bot that cleans up old issues that do not have activity.

This issue has not received feedback in the last six months. I am going to add the inactive label to this issue. If this is still a pertinent issue, please add a comment or add an emoji to an existing comment.

I will post again in five months with another reminder and will close this issue on it's birthday unless it has some activity.

[jlaura]

@jessemapel Looks like you removed the inactive label, but didn't post anything else about why this issue should not be considered inactive. Looks like the bot is not checking to see if the label has been removed, so perhaps that is an enhancement to the bot. I almost like that we can't just remove the label though and that we need an additional post describing why this issue is still pertinent.

[jessemapel]

With CentOS6 EOL in just a few months, this is still something to consider. I am currently drafting an RFC for this.

[jlaura]

@jessemapel What is the status of the RFC for this issue?

[jessemapel]

I was incorrect about GCC4 support being for CentOS6, it was for CentOS7 which is still supported until 2024. I can continue working on an RFC, but I imagine the same institutions that were concerned about this before will be again. @KrisBecker Does LPL still run on CentOS7?

[KrisBecker]

Yes, here are the config tags:

ipwg-8[999]: more /etc/redhat-release CentOS Linux release 7.9.2009 (Core)

ipwg-8[1000]: gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/lto-wrapper Target: x86_64-redhat-linux Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array --disable-libgcj --with-isl=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/cloog-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux Thread model: posix gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC)

We would really appreciate continued support for GCC 4 until we can get our code base added/updated to the current ISIS4+ infrastructure.

Thanks...

[KrisBecker]

Greetings...

Wanted to provide an update on OS support as we received some information from our High Performance Computing (HPC) admins. They will be upgrading to and standardizing UA compute clusters on CentOS7. Here is the rationale:

Why CentOS7 and not CentOS8? IBM surprised the world by announcing CentOS8's conversion into Stream (thanks 2020). Stream will constantly evolve and thus require more frequent OS updates beyond the four scheduled maintenance days we currently have a year. CentOS7 currently has a longer support life than CentOS8, and we are working with NVIDIA to understand how this will impact their CUDA support of CentOS7. While we wait to see what community/distro rises up to replace CentOS (e.g. Rocky Linux) we have decided to go with CentOS7 to minimize impact and downtime for the UA research community. Stay tuned: the situation changes daily, and we are ever vigilant.

There was no mention about GCC 4, but I would assume that is the default compiler version for that OS. I will pursue more info regarding compilers and let you know.

So we would like to implore you all to continue ISIS support for the CentOS7 distro and GCC 4 for awhile longer.

Thanks...

[github-actions[bot]]

Thank you for your contribution!

Unfortunately, this issue hasn't received much attention lately, so it is labeled as 'stale.'

If no additional action is taken, this issue will be automatically closed in 180 days.

[acpaquette]

Regarding building on CentOS, we are running into issues with maintaining the ability to both update ISIS's dependency tree and building on CentOS in general. The main culprits are packages that depend on CentOS specific compiler flags and packages that are being built with newer versions of C++, namely packages like PCL that require C++14 to be linked against.

Some packages that we are maintaining for CentOS build support are: ALE, CSM, and GEOs. If we want to try to update OpenCV we will need to maintain CentOS specific versions ourselves for versions of OpenCV >= 4.5.1. I imagine that this will only get worse as time goes on, with more libraries dropping CentOS support either intentionally or unintentionally.

Their is also an issue with some packages like OpenCV versions >= 4.5.3 where there are specific code changes the prevent the environments from diverging like they are now. To some degree we have been able to take advantage of package version differences that support CentOS but don't require code changes. This is a limited case but is something to consider.

Other packages built with c++14 features like PCL versions >= 1.10.0, aren't possible to update on our current CentOS systems. From my understanding current CentOS systems are stuck with c++11 compilers, hence limiting package updates that depend on features from newer c++ complier versions.

Much of this has been a serious hassle when trying to update from python 3.6 to at least python 3.7.11 in #4547

The two options I can think of to solve this problem are:

• Force the Ubuntu, Fedora, and Mac builds to conform to the CentOS builds. Limiting how much we can update the packages within ISIS
• Drop building support for CentOS

Continuing to build on CentOS adds more overhead for devs since we'll need to push out CentOS builds for various packages listed above. It will also limit how much we are able to update the packages within ISIS. If we continue with CentOS support we will only be able to update our build environment to a point. I am not exactly sure what that point is but mapping out the OS specific limitations of package versions available via conda-forge is tedious and difficult. There is time slotted to take a finer look at the ISIS dependencies coming this next fiscal, so figuring out exactly how far the CentOS build environment can be pushed could be done then.

I have been able to create CentOS environments that build ISIS with python=3.7.11. While this is sufficient for what #4547 intends to accomplish it will lead to more dependency problems in about 2 years when support for python 3.7.11 will be ending. More importantly the CentOS environment is very fragile and has a fair but of hard pinning while the more general environment would let us update continuously rather than these yearly chunks. If we can go with supporting CentOS builds for another few months that is doable, but supporting for years to come will become increasingly difficult.

[jlaura]

I am going to advocate that the current builders (ASC employees) drop support for CentOS because of the problems with supporting the aforementioned dependencies. The default GCC compiler is quite old on those systems.

From what I can see from some googling, CentOS8 has GCC8.x installed, so another option might be to have folks wanting to build on CentOS upgrade. I realize 7.x is not EOL until 2024, but GCC4 was release 14 years ago...

Anyone in the community is welcome to support dependencies that will be fully functional with gcc4 and we can provide support releasing once these dependencies are available via some channel, whether those are instructions and push rights on anaconda cloud, data necessary for the build, etc.

[KrisBecker]

Well, that's unfortunate.

UA just upgraded its HPC cluster to CentOS 7. We have used this cluster extensively and planned to continue to use it.

However, there are several compiler options supported as seen here:

Perhaps we can set up a CentOS 7 system with an upgraded compiler environment or find a build that will run on the HPC?

[jlaura]

@KrisBecker As far as I know the built code should run fine on there. I'm using a standard ISIS release on a CentOS7 system. I am not building ISIS on a CentOS7 system though.

[jessemapel]

We would be dropping GCC4 support for compiling but you can run it on a system that uses gcc4 without issue.

@KrisBecker from your list, any of the gcc 5.4+ compilers should work as they are past the C++11 ABI change.

[KrisBecker]

Ok, that's all good. Thinking out loud here, but I don't think using a newer compiler on CentOS 7 will help as I suspect this creates a real problem with Anaconda package dependencies for building ISIS.

We will eventually run some tests to see if we can find a working version that runs on the HPC.

Thanks...

[jlaura]

@KrisBecker Just to close the loop. And maybe this issue? I am running ISIS4.x, 5.x, and 6.0 on a CentOS 7 system currently. uname -r returns 3.10.0-957.12.2.el7.x86_64 which I am confident is a CentOS 7. So no running issues as the ISIS dependencies are all encapsulated (except some low level graphics drivers) inside of the anaconda environment.

As @jessemapel noted, I can't build ISIS because of the gcc version, but I do not attempt builds on the cluster anyway.

Let us know if that is good and one of us can close this issue.

[KrisBecker]

I think this issue can be closed.

Thanks for your suggestions/help.