Seeding new_fractal_cluster_model

[launerbie]

I'm trying to create 100 fractal clusters with new_fractal_cluster_model(), with the following piece of code (makefractals.py).

from amuse.ic.fractalcluster import new_fractal_cluster_model

for i in range(100):
    print(i)
    model = new_fractal_cluster_model(N=100, random_seed=i)

but running makefractals.py however, throws the following exception:

laub @ F036961: ~% amuse makefractals.py 
0
1
[...]
43
44
Traceback (most recent call last):
  File "makefractals.py", line 7, in <module>
    model = new_fractal_cluster_model(N=100, random_seed=i)
  File "/home/laub/software/amuse-10.0/src/amuse/community/fractalcluster/interface.py", line 272, in new_fractal_cluster_model
    return uc.result
  File "/home/laub/software/amuse-10.0/src/amuse/community/fractalcluster/interface.py", line 250, in result
    particles = self.new_model()
  File "/home/laub/software/amuse-10.0/src/amuse/community/fractalcluster/interface.py", line 241, in new_model
    generator.generate_particles()
  File "/home/laub/software/amuse-10.0/src/amuse/community/fractalcluster/interface.py", line 189, in generate_particles
    result = self.overridden().generate_particles()
  File "/home/laub/software/amuse-10.0/src/amuse/support/methods.py", line 107, in __call__
    result = self.method(*list_arguments, **keyword_arguments)
  File "/home/laub/software/amuse-10.0/src/amuse/support/methods.py", line 107, in __call__
    result = self.method(*list_arguments, **keyword_arguments)
  File "/home/laub/software/amuse-10.0/src/amuse/support/methods.py", line 196, in __call__
    return self.method(*list_arguments, **keyword_arguments)
  File "/home/laub/software/amuse-10.0/src/amuse/rfi/core.py", line 106, in __call__
    raise exceptions.CodeException("Exception when calling function '{0}', of code '{1}', exception was '{2}'".format(self.specification.name, type(self.interface).__name__, ex))
amuse.support.exceptions.CodeException: Exception when calling function 'generate_particles', of code 'FractalClusterInterface', exception was 'lost connection to code'
--------------------------------------------------------------------------
mpiexec has exited due to process rank 0 with PID 8138 on
node F036961 exiting improperly. There are three reasons this could occur:

1. this process did not call "init" before exiting, but others in
the job did. This can cause a job to hang indefinitely while it waits
for all processes to call "init". By rule, if one process calls "init",
then ALL processes must call "init" prior to termination.

2. this process called "init", but exited without calling "finalize".
By rule, all processes that call "init" MUST call "finalize" prior to
exiting or it will be considered an "abnormal termination"

3. this process called "MPI_Abort" or "orte_abort" and the mca parameter
orte_create_session_dirs is set to false. In this case, the run-time cannot
detect that the abort call was an abnormal termination. Hence, the only
error message you will receive is this one.

This may have caused other processes in the application to be
terminated by signals sent by mpiexec (as reported here).

You can avoid this message by specifying -quiet on the mpiexec command line.

I've also tried randomizing the seeds like so (makefractals2.py):

from amuse.ic.fractalcluster import new_fractal_cluster_model

import random

random.seed(1)
seeds = [random.randint(1,10000) for i in range(1000)]

for i, s in enumerate(seeds):
    print(i,s)
    model = new_fractal_cluster_model(N=100, random_seed=s)

This will throw the same exception after 164 models.

laub @ F036961: ~% amuse makefractals2.py
(0, 1344)
(1, 8475)
[...]
(163, 6486)
(164, 3949)
Traceback (most recent call last):
  File "makefractals2.py", line 10, in <module>
    model = new_fractal_cluster_model(N=100, random_seed=s)
  File "/home/laub/software/amuse-10.0/src/amuse/community/fractalcluster/interface.py", line 272, in new_fractal_cluster_model
    return uc.result
  File "/home/laub/software/amuse-10.0/src/amuse/community/fractalcluster/interface.py", line 250, in result
    particles = self.new_model()
  File "/home/laub/software/amuse-10.0/src/amuse/community/fractalcluster/interface.py", line 241, in new_model
    generator.generate_particles()
  File "/home/laub/software/amuse-10.0/src/amuse/community/fractalcluster/interface.py", line 189, in generate_particles
    result = self.overridden().generate_particles()
  File "/home/laub/software/amuse-10.0/src/amuse/support/methods.py", line 107, in __call__
    result = self.method(*list_arguments, **keyword_arguments)
  File "/home/laub/software/amuse-10.0/src/amuse/support/methods.py", line 107, in __call__
    result = self.method(*list_arguments, **keyword_arguments)
  File "/home/laub/software/amuse-10.0/src/amuse/support/methods.py", line 196, in __call__
    return self.method(*list_arguments, **keyword_arguments)
  File "/home/laub/software/amuse-10.0/src/amuse/rfi/core.py", line 106, in __call__
    raise exceptions.CodeException("Exception when calling function '{0}', of code '{1}', exception was '{2}'".format(self.specification.name, type(self.interface).__name__, ex))
amuse.support.exceptions.CodeException: Exception when calling function 'generate_particles', of code 'FractalClusterInterface', exception was 'lost connection to code'
--------------------------------------------------------------------------
mpiexec has exited due to process rank 0 with PID 10434 on
node F036961 exiting improperly. There are three reasons this could occur:

[cut mpiexec error messages]

Can anybody confirm/reproduce this behaviour of new_fractal_cluster_model?

[ipelupessy]

its caused by the algorithm generating less than the requested number of stars iin some cases - I have a fix: https://github.com/ipelupessy/amuse/tree/fractalcluster_nsmax_fix which I make into a pull request - it may need some backend coding to either try again automatically when calling new_fractal_cluster (or at least warn the user about the problem...

[launerbie]

@ipelupessy : I'd like to confirm 500dc59e1a67399536fe4ba84c383e574dab1c30 indeed fixes it for me.

I don't know what to make of your comment on backend coding though...

[ipelupessy]

well, I mean it fixes the error in the code, but the fractal cluster may now return with less particles than requested, which you may still consider unsatisfactor..the new_fractal_cluster_model function could be adapted to catch this and give an error or try again with a different seed and generate a valid model, what do you think?

[ipelupessy]

I have added an option, match_N [=True by default] to exactly match the requested number of particles..

[launerbie]

[...] but the fractal cluster may now return with less particles than requested

I suppose this is inherent to how the fractal generation algorithm works?

I think calling new_fractal_cluster_model(N=100) should always return 100 particles, you seem to have fixed this with the match_N option. And if I understand correctly, what match_N=True does, is call the fractal generation code again with a different seed until a fractal is made with the required number of particles.

It sounds as if we now don't have the guarantee that if I call new_fractal_cluster_model(N=100, random_seed=s, match_N=True) with S different seeds, I will get back S unique fractals due to possible seed reuse? (Btw, even if this is so, I still think it's quite acceptable)

[ipelupessy]

if it needs to retry, new_fractal_cluster_model continues with the same generator, continuing its internal random sequence (so it will duplicate some other model, but not trivially)..have you tested the new code?

[launerbie]

Yes, it (bc84494f59baf49804224d97ceec78a31aae29bd) works as advertised.

#makefractals3.py
from amuse.ic.fractalcluster import new_fractal_cluster_model

modelsA = []
modelsB = []

for i in range(2000):
    modelsA.append(new_fractal_cluster_model(N=100, random_seed=i, match_N=True))

for i in range(2000):
    modelsB.append(new_fractal_cluster_model(N=100, random_seed=i, match_N=False))

hundredsA = [len(i) for i in modelsA].count(100)
hundredsB = [len(i) for i in modelsB].count(100)

print(hundredsA, hundredsB)

laub @ F036961: ~% am makefractals3.py 
(2000, 1993)
/home/laub/githubs/amuse/src/amuse/support/literature.py:78: AmuseWarning: 

You have used the following codes, which contain literature references:

    "FractalClusterInterface"
        Simon Goodwin & Ant Whitworth (2004, A&A, 413, 929)

    "AMUSE"
        ** Portegies Zwart, S. et al., 2013, Multi-physics Simulations Using a Hierarchical Interchangeable Software Interface, Computer Physics Communications 183, 456-468 [2013CoPhC.183..456P]
        ** Pelupessy, F. I. et al., 2013, The Astrophysical Multipurpose Software Environment, Astronomy and Astrophysics 557, 84 [2013A&A...557A..84P]
        Portegies Zwart, S. et al., 2009, A multiphysics and multiscale software environment for modeling astrophysical systems, *New Astronomy*, **Volume 14**, **Issue 4**, 369-378 [2009NewA...14..369P]

  warnings.warn(prefix + self.all_literature_references_string(), exceptions.AmuseWarning)

[ipelupessy]

ok, will merge