Compile FDS with GNU/OMPI on Debian Stretch

[mrgcfraga]

Hello,

I have been using Ubuntu to compile and run parallel FDS simulations, following the guide in the wiki for the compilation (https://github.com/firemodels/fds/wiki/Compiling-FDS-with-GNU-Fortran-in-Ubuntu-Linux). However, a new cluster I have started to use has Debian instead of Ubuntu, and I am having some problems following this same guide. When I try to add the ppa repository (first step in that guide, by using the command "sudo add-apt-repository ppa:ubuntu-toolchain-r/test") prior to installing GNU Fortran 7.2, I receive the following error message:

gpg: no valid OpenPGP data found.

following which I receive a "Unable to locate package gfortran-7" message when I try to install gfortran. Though I am not very familiar with Debian, I have searched for a solution for this problem and was unable to find it so far.

My OS is Debian 4.9.88 x86_64.

[sbenkorichi]

I will take a look.

[sbenkorichi]

Ok, there seems to be an issue with accepting the repo. Here is a work around that I did to get it installed. Follow these steps and let me know how it goes.

1. First of all, which flavor are you using? Mine is Stretch. To know yours, do : # cat /etc/os-release You should get something like this:

   PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
   NAME="Debian GNU/Linux"
   VERSION_ID="9"
   VERSION="9 (stretch)"
   ID=debian
   HOME_URL="https://www.debian.org/"
   SUPPORT_URL="https://www.debian.org/support"
   BUG_REPORT_URL="https://bugs.debian.org/"

2. Since the ppa package wasn't processed, then you need to remove it. Do: # sudo add-apt-repository --remove ppa:ubuntu-toolchain-r/test
3. Install this software to allow you mange your app sudo apt-get install software-properties-common
4. From the main web of the repo here, I added it manually into my OS, through these steps. a. open your source list with your text editor, I use vim, (make sure to be as a root) # vim /etc/apt/sources.list Scroll down, and add these two lines:

   deb http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial main 
   deb-src http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial main 

Save the file and close it. b. let's add the key of the repo, with this line: # sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 60C317803A41BA51845E371A1E9377A2BA9EF27F c. let's update, # sudo apt-get update d. install gfortran-7 # sudo apt-get install gfortran-7

You can check gfortran version by

# gfortran-7 --version 
GNU Fortran (Ubuntu 7.3.0-16ubuntu3~16.04.1) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

5. Let's create a symbolic link to gfortran-7 (although not a must). # sudo ln -sfn /usr/bin/gfortran-7 /usr/bin/gfortran

   # gfortran --version
   GNU Fortran (Ubuntu 7.3.0-16ubuntu3~16.04.1) 7.3.0
   Copyright (C) 2017 Free Software Foundation, Inc.
   This is free software; see the source for copying conditions.  There is NO
   warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Then continue the installation (I haven't done so myself). If you still have an issue let me know.

[mrgcfraga]

Salah,

My # cat /etc/os-release gave pretty much the same output as yours: PRETTY_NAME="Debian GNU/Linux 9 (stretch)" NAME="Debian GNU/Linux" VERSION_ID="9" VERSION="9 (stretch)" ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"

Following the steps of your post, I was able to install gfortran-7. However, when trying to continue following the guide for compiling and running parallel simulations using GNU Fortran in Linux (https://github.com/firemodels/fds/wiki/Compiling-FDS-with-GNU-Fortran-in-Ubuntu-Linux), I have noted another problem, now in the installation of OpenMPI. When I try to make the installer, make -j 2, I receive the error message: make: *** No targets specified and no makefile found. Stop.

I believe the error is linked to some kind of problem that happened when I tried to configure OpenMPI. When I issued the command # ./configure FC=gfortran-7 CC=gcc-7 --prefix=/shared/openmpi_64 --enable-mpirun-prefix-by-default --enable-mpi-fortran --enable-static --disable-shared the last lines of the prompt read: configure: error: in /home/idmec/openmpi-3.1.0: configure: error: C++ preprocessor "/lib/cpp" fails sanity check See config.log for more details

[sbenkorichi]

Ok, good. Let me try it later on my box and see how it goes.

[saschagottfried]

@mrgcfraga Debian Stretch offers GFortran-6 and OpenMPI 2.0.2 for compiling Fortran MPI applications. This will sufficiently compile FDS sources. Would you mind to compile FDS sources with stable packages? I think there is no (urgent) need for setting up another (unstable/testing) toolchain. This might make building a cluster way easier. I thought using stable stuff is a main motivation for using Debian.

Did you try to use the official FDS 6.6.0 Linux bundle (compiled with Intel Compiler (w/ Intel MKL), distributed with Intel MPI) on Debian Stretch? I recently noticed that FDS executables compiled with GNU-7/OpenMPI-3 toolchain are roughly 10% slower.

Strictly following the wiki page for Ubuntu might not lead to a working cluster. What makes you think you are better of with GFortran-7 and OpenMPI 3.1.0 on Debian that justifies the installation issues? Are you interested in a minimal working solution on Debian Stretch?

[mrgcfraga]

@saschagottfried

I don't think my Debian has OpenMPI 2.0.2 installed. How could I verify it?

As for the official FDS 6.6.0, I am interested in using a version of FDS slightly modified by me, so I would need to be able to compile the code myself. In principle, I have been using an older version of FDS (6.5.3, if I'm not mistaken), but, if need be, I could apply my modifications to FDS 6.6. However, since I currently don't have access to the Intel Compiler, I imagine I would not be able to compile FDS 6.6 anyway, right?

[sbenkorichi]

@saschagottfried I had gfortran 6 installed, but now we're going with the latest (which was 7) and currently Gnu has released gfortran-8 and gfortran-9 in development phase. So, we're trying to keep up to date after the compilers have updated, and this can help improve the code by adding new features (especially by complying with the Fortran 2008 standard).

[saschagottfried]

Installing libopenmpi2 gives you Open MPI runtime environment. Installing libopenmpi-dev will give you development support and compiler wrappers like mpifort.

FDS 6.6.0 sources can be compiled with Intel toolchain (Compiler/MKL/MPI) - this is how core developers create the official bundles. You can compile with an open source toolchain as well (GNU/OpenMPI) - requires additional setup for Intel MKL installation to support pressure solvers like GLMAT or SCARC.

[saschagottfried]

@sbenkorichi The creators of Debian Stretch have selected gfortran-6.3 and Open MPI 2.0.2 as a stable environment to compile/run Fortran MPI applications. Unless you explicitly vote against this toolchain/runtime environment it seems to me like a reliable platform for running FDS models on this particular Debian distribution.

Leveraging newer compilers is a good thing. Please add notifications like "removed support for gfortran-6" to upcoming release notes.

[sbenkorichi]

@mrgcfraga I was successfully able to compile Open MPI 3. I haven't faced any issue. My thoughts is that you might have missed to change the /shared/openmpi_64 directory to be written so you can be able to compile in it, that's why the makefile wasn't found. Try this: # chmod -R 0777 /shared # chmod -R 0777 /shared/openmpi_64/

Then try to configure it, if you receive no errors, then proceed to compiling it. Try with both ways here: # make -j 2 or # make then# sudo make install

-- Try it and let me know how it goes.

@saschagottfried Ubuntu itself doesn't have gfortran-7 by default. Except maybe for the latest release 18.4 that might start including it into the synaptic, but that doesn't mean you can't use latest releases of Gnu. Same thing applies for other releases, when 6 was available, on Centos only 4 or 5 was found by default, but one still could install it/ or compile it. Actually gfortran 7 helped by catching quite few bugs in the code, in terms of initialization of certain parameters, where Intel was assuming them to be zeros.

[mrgcfraga]

@sbenkorichi

# chmod -R 0777 /shared/openmpi_64/ yields the following message: chmod: cannot access '/shared/openmpi_64/': No such file or directory

[sbenkorichi]

Did you even create it this directory in the first place? Do this:

# sudo mkdir /shared 
# sudo mkdir /shared/openmpi_64
# sudo chmod -R 0777 /shared 
# sudo chmod -R 0777 /shared/openmpi_64

Then try it again.

[mrgcfraga]

I didn't know I need to create these folders (in all my previous installations, this was not the case). Creating the folders, I was able to chmod them. However, when configuring, I still receive the same errors mentioned in my earlier post.

I think the errors are linked to the C++ compiler and preprocessor (though I don't really know what they are for), because it appears to be in this part of the configuring process that the errors occur.

[saschagottfried]

@sbenkorichi I recognize the value of gfortran-7 for a development use case. But these bugs have been catched and they do not return when compiling with gfortran-6.

This issue is about a end user/customer - probably asking for a "good enough" solution. Using gfortran-6 and Open MPI 2.0.2 on Debian Stretch should work well for @mrgcfraga . Why tinker with a "supposed to be better" custom configuration instead of relying on a "known good set" that just works?

From a user support perspective I would always recommend building a working (cluster) environment with a default toolchain. If that can not be accomplished for any reason, I would not recommend building a custom setup that is even more difficult.

[sbenkorichi]

@mrgcfraga It's hard from my end now to help. It should be straight forward process. As for the folders, you need them, because that's what you are specifying in your configuration destination folder. I think your issue is to do with the ./configure process that's why the ./make isn't working for you. Let's try some debugging, Let's do some updates and upgrades,

# sudo apt-get install build-essential
# sudo apt-get update -y && sudo apt-get upgrade -y

check if you have gcc-7 installed by # gcc-7 --version or# which gcc-7

First of all do me this:

# cd /
# ls -l

paste the line of the /shared , then

# cd /openmpi_64 
# ls -l 

paste the line of /openmpi_64

make sure to remove any files exists in openmpi_64 Then, go to OpenMPI 3 folder that you have extracted, and delete it. Re extract it again.

# cd openmpi-3.0.0
#  ./configure FC=gfortran-7 CC=gcc-7 --prefix=/shared/openmpi_64 --enable-mpirun-prefix-by-default --enable-mpi-fortran --enable-static --disable-shared

At the end, past what it gives you, so I can see this error. Do the same thing with # make and # sudo make install

Try also,# which mpirun and # which mpifort Let me know how it goes.

[mrgcfraga]

It seems to me that OpenMPI was successfully installed now. # which mpirun and #which mpifort don't give anything in the prompt, but I imagine it is because I need to change the .bashrc file accordingly. I will try this tomorrow and get back to you.

[saschagottfried]

I wrote a tutorial in a wiki page to demonstrate my recommended approach. It works out-of-the-box as expected. Hope you find your way.

• Compile FDS with GNU Fortran & Open MPI on Debian Stretch (9.4)

Get familiar with GNU/OMPI toolchain, add your code changes to current master branch and/or particular releases like FDS 6.6.0. Get back here as soon you need help related to adding Intel MKL to this toolchain, although I added some notes that might help interested readers.

Why don't you make your code changes public in a forked repo? I would encourage you to think about this option seriously.

[saschagottfried]

@mrgcfraga Please change the issue title to something like "Compile FDS with GNU/OMPI on Debian Stretch" to emphasize that the problem is related to this Linux distribution.

[mrgcfraga]

@sbenkorichi Following you instructions, I was successfully able to compile and run FDS.

@saschagottfried I did not have an opportunity yet to test all the steps in the guide recommended by you, but I will do so as soon as possible and get back to you.

[sbenkorichi]

@mrgcfraga Ok, glad you sorted it.

@marcosvanella when you have time, is it possible to extend the wiki page to account for Debian as well. PS: you might change 7.2 to 7.3 in the wiki page, or can we just say 7.x (where x is the minor release), so you can avoid updating it each time. Thanks !

[marcosvanella]

Hi Salah and Sascha, I'll add a new wiki page for this Debian distribution. Btw, check if you have permission now to edit wikis. I think Randy gave permission to contributors not long ago.

Sascha, I understand your argument for using the provided packages with the Linux distribution used. Thing is that, for each distribution and distribution version the packages (gfortran, openmpi) versions are different (some gfortran versions dating back to 4.2) and as you know, might lead to different issues. As Salah said, we are trying to restrict these (specially gcc, gfortran) to what we use in our computers and test the code with. The guides we write for Linux compilation are just guides for people who need detailed directions on what to do, trying to provide a way to compile a code with the tool chain versions we are using, and with the objective of avoiding further issues down the road. Advanced users can of course do whatever they see more appropriate. Anyways, this strategy is not written in stone so we are open to alternatives and taking advantage of simplifications. For example, if you have successfully tested fds with this default Debian version X tool chain, we can add a paragraph and a link to your wiki.

[sbenkorichi]

Thanks Marcos, Actually, I noticed that I had access few weeks ago. But, didn't want to edit that page as you might have a better way of doing it. Thanks !

[marcosvanella]

Salah, if you want you can try going ahead an making a wiki page for Debian similar to the Ubuntu one. Otherwise I can make a draft with your instructions later. If Sascha is ok with this, we can make a paragraph right up top on his alternative using default libs for Debian 4.9.X and sending the users of this distribution to his excellent wiki. Let me know.

[sbenkorichi]

I don't have permission now to edit wikis, looks like it has turned to previous settings. Feel free to edit it yourself when you have spare time.

[marcosvanella]

Well, it seems we weren't able to give collaborators permission to edit wikis without giving permission to every github user, so we had to revert permissions. If you guys know a way to set settings in GitHub to give granular permission to collaborators to edit the firemodels wikis (in particular you @sbenkorichi @saschagottfried @SusanKilian ) , we want to hear it. Thank you!

[saschagottfried]

I advocated to use the builtin Fortran/MPI software stack shipping with Debian stretch since it is neither deprecated nor buggy but "good enough". It seemed to me like the most obvious thing to prefer the "simplest possible thing".

From a user and support perspective it should be quite tempting to build and run FDS in parallel with the least amount of modification of environment variables like PATH, LD_LIBRARY_PATH, OPAL_PREFIX effectively reducing things that can go wrong - true more than ever if you can easily add support for advanced communication fabrics (or die trying while having issues).

My takeaway today is that more documentation is required to improve the situation for developers, integrators and end users to make informed decisions. I suggest the community

• maintains a list of supported compilers / MPI runtimes (tabular format - each cell links to dedicated instructions to compile)
• maintains a list of supported platforms (tabular format - each cell links to dedicated instructions to install)
• runs continuous integration efforts/suites covering supported compilers and MPI runtimes (as much as possible with help from collaborators and external CI services like Travis-CI etc.)

There is a difference between a build server that creates build artifacts supposed to run both on older and newer Linux OS flavors and a recent Linux distribution of some guy who wants to compile source code for any reason. Surely there is need for "get up and running" docs covering machines running Ubuntu 18.04, CentOS 7.4 and Debian Stretch using built-in and/or custom-compiled, community-validated software stacks to compile and run FDS. This is definitely a community task.

During the next months I will continue to support such issues to learn, verify issues in isolation and to start/improve suggested documentation.

[bwklein]

@marcosvanella Go to GitHub Project Settings and the 'Options' left menu item. Check the box in the Features section to "Restrict editing to collaborators only ". Then go to the "Collaborators" left menu item and add all the people who you want to be a collaborator for the project.

[marcosvanella]

Hi Sascha, thank you for the input. I agree with you. In cases of newer Linux versions going with the distribution provided software stacks would save users (and us) the complication of compiling these themselves. As long as these have been checked to be working correctly, it would be beneficial to point users of these Linux distributions to simplified package installs.

I also think that maintaining a list of supported tool chains and platforms is a good idea, but has to be a community effort. We are not OS experts and are already stretched (no pun intended) as it is. This applies specially to testing fds with different libraries and platforms in a continuous integration setting.

Let's see if we can give you guys access to editing the wikis. I think that's the first step. This way you can add "how to" pages, etc., in the future for other systems as issues are hammered down.

Bryan, thank you for your note. @rmcdermo please take a look at Bryans post. Thank you.

[rmcdermo]

@bwklein Thanks, but this does not seem to solve the problem for wikis. Here are the settings we have:

When I open up wikis to editing, it opens up editing for all GitHub users, which is not what I want.

[bwklein]

Here is what I would do to lock down Wiki edits to only people on a team that has Write privileges. https://www.screencast.com/t/NaGrjb5p2j

[rmcdermo]

@bwklein Thanks for the video. But actually I understand that part. The question at hand is how to give write access to wikis only, without giving write access to the files in the project repo. When we invite collaborators, like Susan Kilian, they get read access to the repo and one of the team members (me or Marcos) must review and merge their pull requests. But the read level of collaboration is still helpful because we can then assign Issues to these collaborators.

I have a ticket in to GitHub support to see if they have a solution. At the moment, I am resigned to not being able to give write access for collaborators.

[bwklein]

That's what the branch protection feature is for, they can write to wikis, but you restrict the ability to push to branches.

[saschagottfried]

I added the tutorial "Compile FDS with GNU/OMPI on Debian Stretch" to the wiki of this repository.

@mrgcfraga Please close the issue if you have got a solution compiling sources either way covered in wiki pages. Your suggestions what can be added or made more clear are welcome.