search

dev-cafe / autocmake

CMake plugin composer.
http://autocmake.org
BSD 3-Clause "New" or "Revised" License
41 stars 18 forks source link

Ensure that CMake and Python frontend script generate the same build system #225

Closed robertodr closed 6 years ago

robertodr commented 6 years ago

As discussed in #219 we need to make sure that using bare CMake and the frontend script gives the same result. i admit that using the frontend script should be preferred: it can be more concise and actually saves the setup command for later use. However, some users might be already very familiar with bare CMake and we need to make sure not to confuse them.

The strategy here is to make sure that options defined in modules always have a default value. The Python frontend script defines all such options with sensible defaults, the mechanism for this is the define section on top of the module files. This has to be replicated for bare CMake and I've added three macros (thanks @loriab and @ryanmrichard 🎉) to the top of the autogenerated root CMakeLists.txt that can be used for the purpose:

I have fixed some bugs and "fortified" math_libs.cmake to illustrate how this works. Basically, any -DTHIS and -DTHAT that appear in the define sections need to be also listed in the CMake script using option_with_print or option_with_default. This is tedious, but saves a lot of WTF moments and imposing one way of doing on users. As a byproduct of hande-qmc/hande#3, detection and linkage of Intel MKL ScaLAPACK works slightly better. ScaLAPACK from other vendors can be set via SCALAPACK_LIBRARIES.

bast commented 6 years ago

Thank you! I like it :-) The one thing I would change is to not build the text in gen_cmake_options_wrappers line by line, but with one long string between triple-quotes. But that is a detail just to make it easier to read and can be patched later and should not block this one.

robertodr commented 6 years ago

I've implemented your suggested change. The string now also does include(CMakeDependentOption). I have no idea why the tests are failing...

bast commented 6 years ago

So what separates us from WIP to "done" is the failing Boost MPI test? We could either try to figure out why the warning is there or adapt the test script to not choke on the warning.

robertodr commented 6 years ago

Yeah, definitely the failing test should be fixed (though I can't reproduce it on my laptop...) I was also hoping to go through the modules and check whether any of them needs "fortification" with the option wrappers.

robertodr commented 6 years ago

I can't find the time to work on "consolidating" the standard modules as I had intended to do. Do you think we could get this merged and fix the rest piecemeal in other PRs?

bast commented 6 years ago

I agree. I anyway prefer smaller PRs. I have restarted the build.

bast commented 6 years ago

If it fails, I shall prioritize the debugging so that we get this change in.

bast commented 6 years ago

I am working on this today in-between things.

bast commented 6 years ago

test_boost_libs passes on my laptop and fails on Travis - argh! What to do? Merge, let it fail and open an issue that documents that we know about the failing test?

bast commented 6 years ago

Alternative is that I go for some fun-times Travis debugging :-)

robertodr commented 6 years ago

I say merge-and-issue. Those modules should be rewritten as a superbuild anyway...

bast commented 6 years ago

Boost issue followed up in #227. Merging.

miroi commented 6 years ago

Hi Rado and Roberto,

....could not fetch //sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.zip/download

some "download demanding" tests must be improved. For instance, I was recommended to build openblas from source, or, how about to make safe web space for downloading all important libraries (openblas, boost) ?

We can not always rely on external providers....

bast commented 6 years ago

I agree, these downloads from SourceForge et al. can be a pain. But do you know a place where one can store tarballs where the place is not a Git repo and not yet another server we need to maintain?

miroi commented 6 years ago

Somewhere

yourwebserver/~bast mywebserver/~miro robertowebserver/~roberto

or http://diracprogram.org/coderefinery_storage

bast commented 6 years ago

I am really looking for a solution that we do not maintain ourselves.

robertodr commented 6 years ago

I have no idea. It is a "vulnerability" of the current scheme of things that we have to live with, I think. I don't want to maintain an FTP server, full stop. The Boost thing was created as a workaround, I don't like it and I would never recommend its usage to anybody. If a library is needed for code A, install it by hand before building by code A.

On Travis you can have build caches, but that's not the answer you guys were looking for.

bast commented 6 years ago

Same here, I am running too many servers already. I only want to reduce the number, never increase.

miroi commented 6 years ago

Folks, Rado,

no new server for maintenance, only our old, good DIRAC server.

see http://diracprogram.org/doc/coderefinery_storage/

and try online command 

 wget http://diracprogram.org/doc/coderefinery_storage/hello
bast commented 6 years ago

But then we entangle two unrelated projects (well they are linked through the people). If "DIRAC" decides to change things, Autocmake will break. So thank you for the suggestion and efforts but for me this is still a self-maintained solution which has own risks.