idaholab / raven

RAVEN is a flexible and multi-purpose probabilistic risk analysis, validation and uncertainty quantification, parameter optimization, model reduction and data knowledge-discovering framework.
https://raven.inl.gov/
Apache License 2.0
217 stars 133 forks source link

[DEFECT] Setting the seed for RNG in Genetic Algorithm does not produce repeatable results #2232

Open cbroman-usnctech opened 8 months ago

cbroman-usnctech commented 8 months ago

Thank you for the defect report

Defect Description

Setting the sampler seed within Optimizers/samplerInit/initialSeed and Samplers/MonteCarlo/samplerInit/initialSeed do not produce repeatable results.

Steps to Reproduce

Below is the template for repeating the problem. The solution changes every time it is run, despite setting the seed for the RNG in both the model and the sampler.

<?xml version="1.0" ?>
<Simulation verbosity="debug">
  <TestInfo>
    <author>user</author>
    <created>2023-12-18</created>
    <description>
       A template for running the genetic algorithm.
    </description>
  </TestInfo>
  <RunInfo>
    <JobName>analytic-optimization-coefficients</JobName>
    <Sequence>analytic-optimization-coefficients,print</Sequence>
    <WorkingDir>.</WorkingDir>
    <batchSize>10</batchSize>
  </RunInfo>

  <Steps>
    <MultiRun name="analytic-optimization-coefficients" clearRunDir="False">
      <Model class="Models" type="Code">my_raptor</Model>
      <Optimizer class="Optimizers" type="GeneticAlgorithm">genAlgo</Optimizer>
      <SolutionExport class="DataObjects" type="PointSet">opt_export</SolutionExport>
      <Output class="DataObjects" type="PointSet">optResults</Output>
      <Output class="DataObjects" type="PointSet">results</Output>
      <Output class="OutStreams" type="Print">opt_export</Output>
    </MultiRun>
    <IOStep name="print">
      <Input class="DataObjects" type="PointSet">opt_export</Input>
      <Input class="DataObjects" type="PointSet">results</Input>
      <Output class="OutStreams" type="Print">opt_export</Output>
      <Output class="OutStreams" type="Print">results</Output>
    </IOStep>
  </Steps>
  <Models>

  </Models>
  <Optimizers>
    <GeneticAlgorithm name="genAlgo">
      <samplerInit>
        <limit>10</limit>
        <initialSeed>123456</initialSeed>
        <type>min</type>
        <writeSteps>every</writeSteps>
      </samplerInit>
      <GAparams>
        <populationSize>20</populationSize>
        <parentSelection>rouletteWheel</parentSelection>
        <reproduction>
          <crossover type="onePointCrossover">
            <crossoverProb>0.8</crossoverProb>
          </crossover>
          <mutation type="swapMutator">
            <mutationProb>0.8</mutationProb>
          </mutation>
        </reproduction>
        <fitness type="invLinear">
          <a>1.0</a>
          <b>0.0</b>
        </fitness>
        <survivorSelection>fitnessBased</survivorSelection>
      </GAparams>
      <objective>obj</objective>
      <TargetEvaluation class="DataObjects" type="PointSet">optResults</TargetEvaluation>
      <Sampler class="Samplers" type="MonteCarlo">MC_samp</Sampler>
      <variable name="Xwidth">
        <distribution>XwidthDist</distribution>
      </variable>
      <variable name="Ywidth">
        <distribution>YwidthDist</distribution>
      </variable>
    </GeneticAlgorithm>
  </Optimizers>
  <Samplers>
    <MonteCarlo name="MC_samp">
      <samplerInit>
        <limit>20</limit>
        <initialSeed>123456</initialSeed>
      </samplerInit>
      <variable name="Xwidth">
        <distribution>XwidthDist</distribution>
      </variable>
      <variable name="Ywidth">
        <distribution>YwidthDist</distribution>
      </variable>
    </MonteCarlo>
  </Samplers>
  <Functions>
  </Functions>
  <Distributions>
    <Uniform name="XwidthDist">
      <lowerBound>0.0</lowerBound>
      <upperBound>1.0</upperBound>
    </Uniform>
    <Uniform name="YwidthDist">
      <lowerBound>1.0</lowerBound>
      <upperBound>4.5</upperBound>
    </Uniform>
  </Distributions>
  <VariableGroups>
    <Group name="invars">Xwidth,Ywidth</Group>
    <Group name="outvars">targetVal,time,hdfFile,obj</Group>
    <Group name="all">Xwidth,Ywidth,obj,targetVal,time</Group>
  </VariableGroups>
  <DataObjects>
    <PointSet name="results">
      <Input>Xwidth,Ywidth</Input>
      <Output>hdfFile,obj,targetVal,time</Output>
    </PointSet>
    <PointSet name="optResults">
      <Input>Xwidth,Ywidth</Input>
      <Output>obj,targetVal,time</Output>
    </PointSet>
    <PointSet name="opt_export">
      <Input>trajID</Input>
      <Output>all,age,batchId,fitness,iteration,accepted,conv_objective</Output>
    </PointSet>
  </DataObjects>
  <OutStreams>
    <Print name="results" dir="analytic-optimization-coefficients">
      <type>csv</type>
      <source>results</source>
      <what>input,output</what>
    </Print>
    <Print name="opt_export" dir="analytic-optimization-coefficients">
      <type>csv</type>
      <source>opt_export</source>
      <clusterLabel>trajID</clusterLabel>
      <what>input,output</what>
    </Print>
  </OutStreams>
</Simulation>

Expected Behavior

That setting the seed for the RNG would produce repeatable results.

Screenshots and Input Files

No response

OS

Linux

OS Version

No response

Dependency Manager

PIP

For Change Control Board: Issue Review

For Change Control Board: Issue Closure

wangcj05 commented 8 months ago

@JunyungKim Could you take a look at this issue?