randomReal produces cyclic sequence when compiled with MPI

[johnfgibson]

Describe the bug When compiled with MPI, channelflow's randomReal(Real a, Real b) produces a cyclic sequence of numbers with period 12. Without MPI, the sequence is pseudorandom as expected.

The code for randomReal() in mathdefs.h seems pretty suspect. Every call to this function resets the drand48 seed. Is the call to srand48 supposed to be inside an else following the if (&initialized)?

inline Real randomReal(Real a, Real b) {
#ifdef HAVE_MPI
    int taskid = 0;
    int initialized = 0;
    MPI_Initialized(&initialized);
    if (initialized)
        MPI_Comm_rank(MPI_COMM_WORLD, &taskid);
    srand48((long int)(taskid + 1) * 1000 * drand48());
    return a + (b - a) * drand48();
#else
    return a + (b - a) * drand48();
#endif
}

Actual Result Here's a sequence of 40 calls to randomReal() with and without MPI (the latter done by inlining the non-MPI code in the loop, rather than recompiling channelflow without MPI). I added // cycle comments to mark where the cycle starts and when it recurs.

gibson@sophist$ mpirun -n 5 ./randomRealBug.x 
channelflow randomReal() with MPI

0.1708280361062897
0.4017572266320748
0.1223546649134768
0.9441901488035036
0.9982854049458219
0.7734715529522909
0.2305451771981133
0.4945624448164416
0.7832349621030552
0.6732204145091636
0.3915003638985617  // cycle starts
0.9488251265875114
0.8541958730938894
0.659659155718149
0.233356119672866
0.6191251583668453
0.6163142158920927
0.491751502341689
0.4609809141411496
0.61350327341734
0.1694974543797834
0.401263773049326
0.3915003638985617 // cycle
0.9488251265875114
0.8541958730938894
0.659659155718149
0.233356119672866
0.6191251583668453
0.6163142158920927
0.491751502341689
0.4609809141411496
0.61350327341734
0.1694974543797834
0.401263773049326
0.3915003638985617 // cycle 
0.9488251265875114
0.8541958730938894
0.659659155718149
0.233356119672866
0.6191251583668453

channelflow randomReal() without MPI (inlined)

0.6348278058639494
0.6994127725086159
-0.8726713408778863
0.644541587937205
0.3472290862991656
0.2211617541440134
0.8564744741544672
-0.832009674533019
0.6354842052110143
-0.8331626732510031
0.7450901171932429
0.5338898962963654
-0.6300521030320567
-0.6570945813719007
0.3261861127686956
0.5223858872792633
-0.6565840404963978
-0.5523087008506096
-0.4712164414504372
0.7165705539698948
0.1023596516834857
0.1766794333955701
-0.8206981425534678
0.6539392157869273
-0.3742995009555443
-0.7750996554239364
0.3856347634383184
-0.8477776657531635
-0.944558829708825
0.9380128559347369
-0.6948450054311621
0.8459039470265068
-0.7155721352344813
0.3800246175792594
0.3556955889272473
0.5012694182127362
0.7646327492495573
0.1191402714120429
0.2999627345983953
-0.4208912707435317

Steps to reproduce the issue

The code that produces the above output

#include <iostream>
#include "channelflow/dns.h"

using namespace std;
using namespace chflow;

// Illustrate cyclic bug in channelflow randomReal() compiled with MPI 

int main(int argc, char* argv[]) {
    int taskid = 0;
    CfMPI* cfmpi = NULL;

#ifdef HAVE_MPI
    cfMPI_Init(&argc, &argv);
    {
        cfmpi = &CfMPI::getInstance();
        taskid = cfmpi->taskid();
#endif

    const int Ntests = 40;
    cout << setprecision(16);

    cout << "channelflow randomReal() with MPI\n" << endl;
    for (int n=0; n< Ntests; ++n)
      cout << randomReal() << endl;

    cout << "\nchannelflow randomReal() without MPI (inlined)\n" << endl;
    Real a = -1;
    Real b = 1;
    for (int n=0; n< Ntests; ++n)
      cout <<  a + (b - a) * drand48() << '\n';
    }

#ifdef HAVE_MPI    
    cfMPI_Finalize();
#endif
    return 0;
}

Information on your system

Channlelfow compiled and run on openSUSE LEAP 15.0 on an Intel x86-64 architecture with g++-7.3.1 and CMAKE comfiguration

gibson@sophist$ cmake -DCMAKE_CXX_COMPILER=/usr/lib64/mpi/gcc/openmpi/bin/mpic++  -DCMAKE_INSTALL_PREFIX=~/channelflow-master -DWITH_SHARED=ON -DWITH_HDF5=ON ~/gitworking/channelflow

1. platform you are running on
2. relevant configuration details (CMake configuration, etc.)

Additional context

Add any other context about the problem here.