NW doesn't use auxiliary RNG when run from monitor and plot widgets

[qiemem]

See: https://stackoverflow.com/questions/28038007/strange-random-seed-behaviour-after-create-breeds-with

[qiemem]

To reproduce, open a model and stick this in the code tab:

extensions [ nw ]

to testRNG
  clear-all

  random-seed 10

  type random 100
  type ","

  create-turtles 10 [ fd random 10 ]

  type random 100
  type ","

  ask turtles [ create-links-with other turtles in-radius 5 ]

  print random 100
end

Create a monitor with the code:

length nw:weak-component-clusters

Turn the speed of the model down a bit and run repeat 10 [ testRNG ]. It should produce the same output every time, but instead produces something like:

78,79,16
78,79,98
78,79,72
78,79,98
78,79,66
78,79,66

[qiemem]

Oddly, it doesn't happen with nw:distance-to.

[digitaldust]

I am using the NW extension only with nw:distance-to in the code, I have some monitors that just counts agents but I am experiencing the same issue, my runs are not reproducible no matter specifying a random-seed

[digitaldust]

just to know, no one is assigned to this bug, so is this something you are investigating? if not, kindly let me know...

[qiemem]

I am using the NW extension only with nw:distance-to in the code, I have some monitors that just counts agents but I am experiencing the same issue, my runs are not reproducible no matter specifying a random-seed

Would you post the code in your monitors and the parts of your code that use nw?

just to know, no one is assigned to this bug, so is this something you are investigating?

I'm implicitly assigned to the bug.

[digitaldust]

sure! for monitors basically this line, with other link breeds and other religion values count diffusions with [[religion] of end1 = "muslim" and [religion] of end2 = "muslim" ]

for the code, it's simply let dist nw:distance-to myself inside go and nw:set-context turtles parentals within setup

but is the nw extension using another RNG that I have to seed apart from the main NetLogo RNG?

[qiemem]

Thanks!

but is the nw extension using another RNG that I have to seed apart from the main NetLogo RNG?

Shouldn't be.

If you remove the monitors, does the problem go away?

[digitaldust]

no it doesn't

[digitaldust]

btw, I suspect that something's wrong with random-seed in NetLogo... I have added this line to the model

random-seed item behaviorspace-run-number seeds

where seeds is a list of numbers... if I try to run the model, it works fine although the replications are not exactly the same. But if I try to run a bunch of experiments through BehaviorSpace then I got the error:

org.nlogo.nvm.EngineException: The tick counter has not been started yet. Use RESET-TICKS.

could that be related to this issue of not being able to replicate exactly after seeding the model?

[SethTisue]

Are you able to prepare a SSCCE that demonstrates what you believe the issue to be?

Without that, this just seems like a guessing game. We don't know what you're doing; we don't know what's in your model or your BehaviorSpace experiment. And it's highly unlikely that "there's something wrong with random-seed"; that's the one of last hypotheses you should be considering, only after everything else has been rigorously and carefully ruled out.

Is the problem reproducible outside of BehaviorSpace?

Is the problem reproducible without involving the nw extension?

What is the absolute shortest, minimal .nlogo file that demonstrates the bug, from which literally nothing else can be removed while still demonstrating the bug?

And so on.

[digitaldust]

sure, you are absolutely right, sorry for not being polite on this. Could you try:

extensions [csv]

globals [seeds]
to setup
    ca
    ;; If you use this line and run the model it works
    ;; but with BehaviorSpace you get "The tick counter has not been started yet. Use RESET-TICKS."
    set seeds item 0 csv:from-file "random-seeds.csv"
    ;; this line instead works perfectly in both cases
    ;set seeds n-values 100 [random 1000]              
    random-seed item behaviorspace-run-number seeds
    create-turtles 100 [setxy random-xcor random-ycor set heading random 360]
    reset-ticks
end

to go
    ask turtles [fd 1 set heading random 360]
    tick 
end

and this are the numbers I use in the random-seeds.csv

0, 36494,87011,23158,74099,11483,3174,24577,5787,21246,30393,83557,86291,12820,93581,99619,61555,77282,64965,16085,10680,35984,21244,7696,53907,88041,5057,36880,32747,32446,99076,31068,62928,76890,56768,99818,50318,9228,33096,81658,93219,93670,51031,62666,58216,65246,54862,79052,64895,91648,15380,13037,89688,83881,70714,27444,71396,97673,80651,6423,51297,15293,35553,71968,79285,15096,15865,20311,37949,52031,64518,8036,92131,25677,42621,83037,71843,52390,97374,9808,45872,22040,81365,9253,73816,84698,88615,90090,32240,67574,39090,28611,44115,56196,26102,55929,73617,9929,64511,51031,51544

I really do not understand why the first solution doesn't work and the second does... thanks for your help

[digitaldust]

forgot to mention my system:

NetLogo 5.2.0 (April 3, 2015) Extension API version: 5.0 Java HotSpot(TM) 64-Bit Server VM 1.6.0_65 (Apple Inc.; 1.6.0_65-b14-466.1-11M4716) operating system: Mac OS X 10.10.3 (x86_64 processor) Scala version 2.9.2 Java heap: used = 8 MB, free = 73 MB, max = 1015 MB

[SethTisue]

well, this is indeed peculiar. @qiemem, what do you make of this?

[SethTisue]

It looks like a BehaviorSpace bug to me — some error is occurring during setup, the error isn't being reported, and then because setup didn't finish, go fails.

[SethTisue]

Also, in your random-seeds.csv file, is there a space after the first comma, but not after the other numbers? it appears that the CSV extension is sensitive to whitespace, so I get e.g.:

observer> show csv:from-file "random-seeds.csv"
observer: [[0 " 36494" 87011 ... ]]

Note that a string got in with my numbers, only because of that extra space after the first comma.

[digitaldust]

thanks Seth, that was indeed the problem with the seeds, removing the space solved the issue. I guess that at some point with BehaviorSpace happened:

random-seed " 36494"

but no error was reported and setup, as you said, failed.

Anyway, I am generating links with my model and still I am not able to generate the same number of links using the same seed.

[qiemem]

Can you create a similarly simple model that demonstrates the nondeterministic behavior? I haven't been able to reproduce it.

[digitaldust]

I am trying to do that but I am not able to with a simpler model. As soon as I have something, I will post it. All I can say as of now, is that I am try to identify the chunk of code where I mess with the RNG. My setup procedure is like:

to setup
    random-seed 0
    ;; I do some stuff then
    reset-ticks
    print (word "after setup " random 10)
end

All the time I run setup, I get 3 as an output, so I guess nothing fuzzy happens in setup. What surprises me is what happens in go:

to go
    if ticks mod 100 = 0 [ print (word "before stop " random 10) ]
    if semester = 53 [   print random 10 store-final-outcome stop]
    if ticks mod 100 = 0 [ print (word "before seed " random 10) ]
    seed-potters
    if ticks mod 100 = 0 [ print (word "after seed " random 10) ]
    move
    if ticks mod 100 = 0 [ print (word "after move " random 10) ]
    store-run-results
    if ticks mod 100 = 0 [ print (word "after store " random 10) ]
    tick
end      

This is the output of four different runs:

after setup 3   after setup 3   after setup 3   after setup 3
before stop 7   before stop 7   before stop 7   before stop 7
before seed 0   before seed 0   before seed 0   before seed 0
after seed 2        after seed 2    after seed 2    after seed 2
after move 9    after move 9    after move 9    after move 9
after store 1           after store 1   after store 1   after store 1
*iteration 2*
before stop 5   before stop 1   before stop 1   before stop 1
before seed 3   before seed 7   before seed 3   before seed 7

As far as I understand it, but I might be wrong, it seems something happens after calling store-run-results and before calling again seed-potters, but I only call tick... this is really puzzling me.

As soon as I understand the reason why this happens, I should be able to send a reproducible example.

[SethTisue]

tick triggers code in plots. Does your model have plots?

Code in plots is supposed to use an auxiliary RNG, but maybe there's some bug there, like maybe the nw extension doesn't inherit the right RNG from the plot context.

[digitaldust]

Thanks Seth, I have made a progress (although slowly), and what I found is that if I comment out the move procedure the random sequences are identical. I have code to plot inside store-run-results, so I guess the plot stuff is alright. I will look into the move procedure and report what I found. Thanks for your patience.

[digitaldust]

Guys, unfortunately I am not able to understand what happens, nor to make a reproducible example. The only thing I know is that after "tick" something goes wrong because the output for the random number is different across runs, but the only thing I have in the update procedure of my plot is:

 if ticks > 0 [
     foreach table:keys results [
         create-temporary-plot-pen (word ? "-sim")
        ;; pick one of the colors
        set-plot-pen-color item (position ? religions) rel-colors
        ;; pen mode set to draw line
        set-plot-pen-mode 0
        ;; plot the value for this semester
        plotxy semester (last table:get results ?)
    ]
]

and of course it does not matter if I comment out the whole update procedure in the plot, still I have a different sequence of random numbers after tick

[digitaldust]

it's always after tick that something happens

after setup 4   after setup 4   after setup 4
before stop 8   before stop 8   before stop 8
before seed 5   before seed 5   before seed 5
after seed 6    after seed 6    after seed 6
before move 2   before move 2   before move 2
after move 9    after move 9    after move 9
before store 3  before store 3  before store 3
after store 2   after store 2   after store 2
before tick 3   before tick 3   before tick 3
after tick 0    after tick 0    after tick 9
before stop 1   before stop 1   before stop 8
before seed 8   before seed 8   before seed 2
after seed 8    after seed 8    after seed 9
before move 9   before move 9   before move 5
after move 1    after move 1    after move 6
before store 7  before store 7  before store 5
after store 3   after store 3   after store 7
before tick 5   before tick 5   before tick 7
after tick 9    after tick 7    after tick 4

[digitaldust]

I finally solved my issue getting rid of nw:load-graphml

now I can get all the run to be identical with a single seed, leaving all the other code in my model untouched. I guess that load-graphml does something strange when you have multiple types of links in the file. I am sorry not being able to tell you more things on this, anyway I hope this helps to look in the right place.

[qiemem]

Great, thanks for the update @digitaldust. I'll see if I can track down the problem in nw:load-graphml.

[digitaldust]

@qiemem just a last comment from my side. I tried loading the graphml in my own extension. I either tried to create turtles within the extension or to just load the file as a JUNG graph and the pass a logolist back with nodes attributes and a second logolist with links like [[(turtle 0) (turtle 1) "parental"] [(turtle 0) (turtle 1) "diffusion"]]... no matter what I tried, the RNG was somehow affected and I wasn't able to reproduce two runs with the same seed. I have now converted the graphml into two separate csv files for nodes and links and loaded them with the csv:from-file and everything's ok, runs are reproducible. I don't know if this is something related to my graphml file. I cannot share it publicly here but if you want to try it I can send it to you privately.

[digitaldust]

@qiemem is there any news on this load-graphml issue with RNG?

[qiemem]

Okay, I've managed to fix the original RNG issue and I can't reproduce the load graphml issue so I'm going to close this. @digitaldust if you're still having problem with graphml in NetLogo 6.0 (when it comes out), open a new issue and I'll see what I can do.