Improve the build system for non-default gcc locations

[s-andrews]

I've had to build partis on a CentOS 6 base which has additional newer versions of GCC and CMake. I've managed to get partis to build and run but it was a very manual process because of the limitations of scons. The main problem is that scons seems to reset the environment when it runs, so modified paths or CC/CXX variables get ignored.

The specific places this caused problems were with the ham/yaml_cpp where the cmake call selects the old gcc and then fails to build because it doesn't recognise the gcc options. The same thing happens in packages/ig-sw/src/ig_align where the compiler options cause the stock gcc to fail (-Ofast isn't recognised).

Either an option to set the gcc location, or specifically picking this up from the environment should make this work a lot smoother.

Thanks.

[psathyrella]

I'm not sure I understand which components are causing which problems, which is probably partly because some of them I wrote, and some of them I've never even looked at. So I think scons is (only?) getting used by ig-sw and bcrham, and those I'm pretty familiar with although I only wrote the latter. Anyway, It looks like their SConstruct files are fairly different. Perhaps just adding the os.environ to the Environment constructor in the ig-sw one would solve one of your problems?

Now I haven't used make in a while, but samtools is built with make in bin/build.sh. But I'm not seeing anything using CMake, so I think I'm not looking at the right thing?

[psathyrella]

gah, I'm an idiot, sorry. Right, the cmake in ham/yaml_cpp. I'll start digging if there's a way to reconfigure the cmake in there. I think if nothing else I wanted to update the yaml-cpp code in there, it's been a while.

[psathyrella]

ok, there's actually ubuntu and rhel packages for yaml-cpp. I'm not even remotely enamored of having the darn yaml-cpp code actually inside ham, so maybe we can just nuke it and add the yaml-cpp requirement to the docker file? That appears to require adding boost as a dependency, which sucks, but they say the next yaml-cpp version will not use boost, so maybe not for long.

Together with adding os.environ to the ig-sw SConstruct, would that solve your issues?

[psathyrella]

Note to self: do this when the yaml-cpp version in the packaging systems no longer requires boost.

[williamdlees]

I also ran in to this, and can confirm that adding os.environ to the SConstruct constructor would be helpful

[psathyrella]

ok, I've added os.environ to the ig_sw scons, not sure why I forgot to do that before.

Now assuming you're also having the issue with gcc version in yaml-cpp/cmake, it's less clear to me how to fix it. The boost-less yaml-cpp looks like was just released, but while we're waiting for that to propagate to ubuntu, I'm not sure why the environment isn't getting passed to cmake. I'm calling cmake from within a scons file with os.environ here, which I think should do that. If I tack an env call on the start, anyway, it knows about all my user-defined env variables.

What variable are you setting to specify the non-default gcc location?

[williamdlees]

Must admit I ran out of patience with SConstruct at that point (I've never come across it before). I deactivated conda, went into the _build directory and issued the make commands for yaml-cpp from the console.

The environment variables I use to set up my up-to-date gcc environment are LIBRARY_PATH, LD_LIBRARY_PATH, CC, CXX .

[psathyrella]

ok, thanks for the clues. I just made sure that those variables are getting passed from the calling shell to the cmake command that builds yaml-cpp, and all looks good. But I'm not using conda (and don't have much experience with it), so perhaps it's doing something wonky. In any case, since it sounds like you've got it installed I don't think I'll spend any more time tracking it down at the moment, especially since the main bug you found was the missing os.environ in the other, ig-sw scons file.

Dear future self: should still switch to the apt version of yaml-cpp when the boost-less version gets that far.

[psathyrella]

switched here to yaml-cpp from apt.