search

E3SM-Project / scream

Fork of E3SM used to develop exascale global atmosphere model written in C++
https://e3sm-project.github.io/scream/
Other
80 stars 55 forks source link

What's stopping us from supporting SCREAM on Mac laptops and desktops? #484

Closed jeff-cohere closed 3 years ago

jeff-cohere commented 4 years ago

It's incredibly helpful to be able to develop in a friendly local environment with comfortable tools and no queues. I know I really enjoy being able to develop, test, and debug SCREAM on my Linux workstation and laptop. But a lot of folks use Apple hardware, and I've heard that we don't plan to add support for those machines.

Is there a reason for this? Are we worried that it's too much work? I know that the way we do floating point exception handling doesn't work on Macs, for example, but I also know how to do that stuff on Macs. Are there other obstacles?

If it's just a matter of legwork, I think it's worth making an attempt to get SCREAM building and running locally on Macs in addition to Linux machines. I think it might make some folks more productive, and it would also give us another set of compilers and environments that may give us more angles from which to understand some of our trickier outstanding (and yet-to-be-encountered) issues.

bartgol commented 4 years ago

I don't have a strong feeling either way. To be clear though, there are two issues:

  1. Making the current SCREAM source code Mac-compatible
  2. Keeping Mac support on newly written code.

Task 1 could be done by some Mac enthusiast that really want Mac support. Task 2 is a continuous task, and it's worth asking "how much time/effort would it cost to every developer to make sure Mac is not broken?", and "is it worth forcing each dev to keep Mac support, or maybe have 1-2 people in charge of maintaining it"?

I have zero interest in a Mac build, but I am not opposed to have someone supporting Mac. Regarding point 2, however, I don't want our developers to have to worry about Mac. Sure, assuming all the infrastructure is in place, an average developer should probably see zero extra work. However, some of us are doing infrastructure stuff, and I don't think they should support an OS that is not among our target ones for production. So, to reiterate, if someone really wants Mac support, they should feel free to add it (possibly on their own time, if PI's do not want SCREAM $$ to go towards this effort), but they should also know that each PR that goes in might break their Mac builds.

jeff-cohere commented 4 years ago

I think that's practical, and I'd be willing to try to take care of Task 1, since I have Mac hardware (but don't rely on it solely). I suspect once we get it working, Task 2 would certainly be work, but it could be as simple as just fixing compiler errors (and might help us clean up our C++ a bit, since it unlocks a different C++ compiler).

bartgol commented 4 years ago

What compiler would you use? Clang?

jeff-cohere commented 4 years ago

Yes, the Apple Clang compiler (with OpenMPI).

bartgol commented 4 years ago

Does it use Flang for F90? I haven't checked out Flang personally, but someone told me it's not yet working great for HPC stuff.

jeff-cohere commented 4 years ago

Flang is still in its infancy, I think, so I use gfortran.

bartgol commented 4 years ago

That is an interesting setting, since it would check that compilation with two different host compilers works correctly. Notice that you might have to change the cmake macro EkatDisableAllWarnings, since it only adds flags for Gnu and Intel compilers.

pbosler commented 4 years ago

@jeff-cohere I put in a CMake check for the lack of feenableexcept for Mac builds in branch pbosler/mac_build_iss484.

It's a starting point, to get ekat_assert.cpp to build.

In scream/components/scream/CMakeLists.txt,

print_var(CMAKE_SYSTEM_NAME)
include(CheckSymbolExists)
check_symbol_exists(feenableexcept "fenv.h" HAVE_FEENABLEEXCEPT)
print_var(HAVE_FEENABLEEXCEPT)

The output on mac is:

CMAKE_SYSTEM_NAME: Darwin -- Looking for feenableexcept -- Looking for feenableexcept - not found HAVE_FEENABLEEXCEPT:

On linux, it's:

CMAKE_SYSTEM_NAME: Linux -- Looking for feenableexcept -- Looking for feenableexcept - found HAVE_FEENABLEEXCEPT: 1

jeff-cohere commented 4 years ago

Rockin. Thanks @pbosler !

bartgol commented 4 years ago

I just found this post on stackoverflow (seriously, what would we do without stackoverflow?), which could help with mac.

jeff-cohere commented 4 years ago

That's good. Also, I've gotten this FPE nonsense to work with Macs before, so I can just bring over some code from one of my own projects if needed.

bartgol commented 4 years ago

Whatever is easiest. While at it, it might also be worth to add a "catch all" case, and make it so that the debug_tools test does not perform FPE checks if the compiler is not one of those for which we implemented FPE support...

bartgol commented 4 years ago

@jeff-cohere @pbosler now that EKAT builds on Mac (albeit still with some issues regarding FPEs), is SCREAM building/running on Mac?

jeff-cohere commented 4 years ago

Good question! I'll give it a try today and update this issue with the results.

jeff-cohere commented 4 years ago

I think we need to bump SCREAM's version of ekat before we can expect this to work. At this point, we've got a lot of changes in ekat, some that require changes to SCREAM code, so I'm not sure when it would be best to do this. @bartgol , do you have thoughts on when we should bring the new Kokkos version into SCREAM?

bartgol commented 4 years ago

I was hoping to finish the generalization of host_to_device and device_to_host in ekat, before bringing the new ekat in. However, given the long weekend, that'd happen next week. If you want to speed things up, we could bring it in now, but we'd need to hack a bit how p3/shoc use the aforementioned functions, since as it is, it doesn't work (that's why I started to generalize them).

Up to you. I can make it a priority when I get back to work on Tuesday, if you're ok waiting.

jeff-cohere commented 4 years ago

I'm in no great hurry, so let's just do it early next week. Appreciate your effort!

PeterCaldwell commented 3 years ago

I think this issue is solved by https://github.com/E3SM-Project/EKAT/pull/21 but it seems like most folks on the project don't know that we can build SCREAM on our macs yet. Am I correct we can close this? Can we create some confluence documentation (or inline documentation in github or something) about how to run scream tests from a mac? @bartgol @pbosler @jeff-cohere ?

bartgol commented 3 years ago

This issue seems stale. I'm closing it. @jeff-cohere @pbosler , you're the main MAC users (AFAIK), so if you'd like to reignite the discussion, feel free to reopen.