TeamCOMPAS / COMPAS

COMPAS rapid binary population synthesis code
http://compas.science
MIT License
64 stars 66 forks source link

Issue with replicating the binary produciton with detailed output #928

Closed syedalimohsinbukhari closed 1 year ago

syedalimohsinbukhari commented 1 year ago

Problem

I generated a set of binaries with a custom metallicity values in a grid file. I passed it to pythonSubmit file with a few options changed inside the file,

  1. evolve-pulsars TRUE
  2. remnant-mass-prescription MULLERMANDEL

After running the file it generated the binaries and I started investigating the DCOs in detailed by using the following parameters in a new grid file

  1. random-seed
  2. initial-mass-1
  3. initial-mass-2
  4. semi-major-axis
  5. eccentricity
  6. metallicity
  7. remnant-mass-prescription
  8. evolve-pulsars

The values were taken of those binary systems that did evolve into DCOs eventually within Hubble time.

Issue

This issue is that the initial generation of the binaries and the subsequent detailed evolution is not giving the same answer, I've been investigating the issue on my own and I've found one factor that is different in both the files

SEED (OPTION).

There are three fields in BSE_System_Parameters that have SEED numbers in them,

  1. SEED
  2. SEED (CMDLINE)
  3. SEED (OPTION)

In the original file, all generated binaries have SEED (CMDLINE) and SEED (OPTION) as the same which comes from np.random.seed(235) (the random seed is 7942948522105630304) that i defined in the pythonSubmit file.

However, for detailed output, although I'm using the exact same pythonSubmit file but the generated binaries do not have same SEED (CMDLINE) and SEED (OPTION) values.

So I'm wondering what can be done to produce similar DCOs, or am I doing something wrong. I've tried running the file in terminal and IDEs as well but the issue persists. I will attach screenshots in a follow-up comment.

syedalimohsinbukhari commented 1 year ago

For example,

image

  1. I read the COMPAS output file,
  2. Store the DCO parameters and System Parameters separately
  3. Search for the first DCO that merges within Hubble time and look all the parameters up in the BSE_System_Parameters
  4. At the end I generate a grid file form these parameters

image

When I generate the binaries using this grid file and same pythonSubmit file, the output is different.

As you can see in the screenshot below, the values of Mass(2), SemiMajorAxis and Eccentricity don't match up.

image

jeffriley commented 1 year ago

Welcome to COMPAS @syedalimohsinbukhari.

What version of COMPAS are you using? We haven't used pythonSubmit.py for a while (we use runSubmit.py and a yaml file now).

I hope I have understood your problem correctly - if not, please reply and let me know where I went wrong.

However, for detailed output, although I'm using the exact same pythonSubmit file but the generated binaries do not have same SEED (CMDLINE) and SEED (OPTION) values.

'SEED (CMDLINE)' will show the random seed as it was type at the command line. If you are using pythonSubmit.py, this will be the random seed specified in pythonSubmit.py.

'SEED (OPTION)' will show the random seed that was entered as an option value in the grid file.

If you use pythonSubmit.py and specify a random seed in phytonSubmit.py, and you specify a grid file in pythonSubmit.py, the random seed for each binary in the grid file will be taken from the option in the grid file if one is specified, and where no --random-seed option is specified in the grid file, then the random seed will fallback to the one specified on the command line (i.e. the one specified in pythonSubmit.py). If I understand the scenario you've described correctly, I would expect those two values to be different (I think you specified a random seed in the pythonSubmit.py file, but had a single entry in the grid file with a different random seed specified?).

To replicate a binary from the first run, and create the correct detailed output, the initial parameters, including the random seed, need to be the same - you will need to specify the kick parameters from the initial run too (these are stochastic, so if you don't take the values from the initial run and pass them to COMPAS for the detailed output run, you will get different kick values).

You don't need to use pythonSubmit to create a single binary from a grid file. You can just run COMPAS from the command line yourself. e.g., using your example:

./compas --random-seed 7942948522105637045 --initial-mass-1 11.129984513185423 --initial-mass-2 8.704449420142403 --metallicity 0.02804375385514442 --eccentricity 0.0 --semi-major-axis 0.5439306779852381 --remnant-mass-prescription MULLERMANDEL --evolve-pulsars

However, as I said, you'll need to specify the kick values that you got from the first run - you should be able to see that they are different in your second run (the detailed output run with the grid file).

syedalimohsinbukhari commented 1 year ago

Thank you @jeffriley for the detailed breakdown, I'll follow it and will let you know about it. As for the COMPAS version, the original files were generated using a previous version that had pythonSubmit so we've kept that for consistency purposes.

jeffriley commented 1 year ago

@syedalimohsinbukhari Have you had a chance to look at this - is it still an issue for you?

syedalimohsinbukhari commented 1 year ago

@jeffriley Hi, sorry for the delay, yes I'm looking into it and I am still facing problem, I will give an update on it soon.

jeffriley commented 1 year ago

@syedalimohsinbukhari any update on this?

jeffriley commented 1 year ago

No activity for a long time - closing this. @syedalimohsinbukhari you can re-open if the problem persists.