charmplusplus / charm

The Charm++ parallel programming system. Visit https://charmplusplus.org/ for more information.
Apache License 2.0
200 stars 50 forks source link

Better CMake support #1098

Closed epmikida closed 6 years ago

epmikida commented 8 years ago

Original issue: https://charm.cs.illinois.edu/redmine/issues/1098


It would be useful for CMake to be able to better handle Charm++ programs. Some of the issue comes from CMake's side, but there are things we could probably do in Charm++ to make things easier.

CMake has no problem recognizing ".C" as a C++ file, so changing the CMAKE_CXX_COMPILER to charmc for the whole project will get all regular source files compiling mostly without issue. I say mostly because if you are working in an existing project that uses stricter warnings, the compilation of the Charm stuff will likely make it fall over. Luckily you can set the flags of individual files, so I was able to sidestep this by adding the -Wno-error flag to the ".C" files I wanted to compile, while keeping the strict warnings for the rest of the project. If Charm begins to see more commercial use though we may want to at least try to cut down on the huge list of warnings we see every time we compile.

The trickier part in getting Charm++ to work with CMake is .ci files. CMake does not recognize .ci files so you have to manually add rules to get compilation working nicely. For a project I'm working on I came up with the following in my CMakeLists.txt:

file(GLOB CHARMSOURCES charm/*.C )

file(GLOB CISOURCES charm/*ci )

  1. Generate a list of the decl and def files we need to create set(CHARMHEADERS) foreach(ci ${CISOURCES}) string(REGEX REPLACE ".ci\$" ".decl.h" decl ${ci}) add_custom_command(OUTPUT ${decl} COMMAND charmc ${ci} DEPENDS ${ci}) set(CHARMHEADERS ${CHARMHEADERS} "${decl}") endforeach(ci)

add_executable(exe ${SOURCES} ${CHARMHEADERS} ${CHARMSOURCES})

Unfortunately with the above code, the decl and defs are not put in the correct place and charmc does not allow -o when compiling ci files, so the command will have to change to call charmc from within the src directory. It may be helpful for future users to work out some of these kinks on our end, and also add some documentation for the best ways to handle Charm++ in CMake so people don't have to re-invent this or something similar every time they want to use CMake and Charm++.

epmikida commented 5 years ago

Original date: 2016-06-10 18:22:57


Here is the relevant bits of a Slack conversation about this topic:

Eric Mikida: is it possible to run charmc on a ci file and specifiy the path where we want the decl and defs to be put?

Phil Miller [11:02 AM] It writes the output in the current working directory. The easy way to get that effect would essentially be to run it in the desired output directory with a full or relative path to the input source

It also has options for output direction and copying. It could be adapted to understand -o or whatever with .ci file input to mean the place it should write those files

I don’t think those options would currently do anything for .ci file processing though

Eric Mikida [11:04 AM] they dont. i tried I'm just trying to figure out the best way to get charm to play nice with CMake. Switching directories before compiling ci files seems like the best bet for now, thanks.

Phil Miller [11:05 AM] This sounds like something we ought to fix in charmc itself, to make future use of CMake easier

It would be very helpful if you could write up a spec or brief description of how CMake would use it and what information it would have available, and stick that in a Redmine issue

I think we ought to just modify it to take -o and do the right thing

Though with multiple output files for each input, that gets a tad sloppy

Could you copy this conversation into said new issue, and we can continue it ‘on record'

?

Eric Mikida [11:07 AM] sure.

Phil Miller [11:07 AM] Star this message Thanks

Sam White [11:14 AM] Star this message You might want to take a look at Jozsef Bakosi’s quinoa project, which seems to use CMake and Charm++: https://github.com/quinoacomputing/quinoa

stwhite91 commented 5 years ago

Original date: 2016-06-10 18:26:48


Making an example or converting an existing one in examples/charm++/ to use CMake might be a good way to illustrate usage and give users a template.

epmikida commented 5 years ago

Original date: 2016-06-10 18:34:54


Yeah. The project you linked has a very nice looking template charm.cmake that defines some rules for adding a charm module as a dependency to a CMake target. If it can be made universal enough we may want to just write our own cleaned up version with a bunch of useful charm rules and distribute it with charm so people can just import it into their CMake projects without having to re-invent the wheel every time.

PhilMiller commented 5 years ago

Original date: 2016-11-29 15:26:06


Other calls for CMake support (based on GMail searching my mail archive): https://lists.cs.illinois.edu/lists/arc/charm/2011-03/msg00004.html https://lists.cs.illinois.edu/lists/arc/charm/2012-07/msg00000.html https://lists.cs.illinois.edu/lists/arc/ppl/2012-07/msg00055.html https://lists.cs.illinois.edu/lists/arc/charm/2013-10/msg00010.html https://lists.cs.illinois.edu/lists/arc/charm/2014-05/msg00003.html And the SNL DARMA crew

This may be one of the single most popular requests we've gotten. Most of those are from applications people, and not whiny CS researchers, too. Maybe worth investing real effort in doing this, and doing it well.

stwhite91 commented 5 years ago

Original date: 2017-04-19 18:16:56


Nils Deppe said he was able to take Quinoa's cmake support for Charm and use that directly: https://github.com/quinoacomputing/cmake-modules/blob/master/charm.cmake

stwhite91 commented 5 years ago

Original date: 2017-04-20 14:32:06


Christoph and Jozsef encouraged us to take the file above and distribute it with charm. The only issue with Quinoa's cmake rule is that it assumes that there is one module per file... And I don't know about licensing

PhilMiller commented 5 years ago

Original date: 2017-06-06 16:20:11


We're going in on this heavily at Charmworks. Assigning to myself for now.

stwhite91 commented 5 years ago

Original date: 2017-08-30 23:10:55


https://charm.cs.illinois.edu/gerrit/#/c/2958/ https://github.com/UIUC-PPL/charm/commit/e8b3ecba490f0ae7f0c0c07eb780a5ab8972cb27

evan-charmworks commented 5 years ago

Original date: 2017-08-31 22:24:24


Regarding the output location of decl and def, what if I added something like -output-dir <dir>? Using the existing -o switch as a destination directory for multiple files would be nonstandard and potentially confusing.

evan-charmworks commented 5 years ago

Original date: 2017-12-22 22:38:37


Is there anything more than https://charm.cs.illinois.edu/gerrit/#/q/project:charm+topic:cmake-wrapper that this issue needs done?

evan-charmworks commented 5 years ago

Original date: 2018-01-16 23:00:46


Bump. These patches are merged: https://charm.cs.illinois.edu/gerrit/#/q/project:charm+topic:cmake-wrapper

Is there anything else needed for this issue?

stwhite91 commented 5 years ago

Original date: 2018-01-16 23:11:57


I don't think so.