search

boettiger-lab / pomdp-intro

:notebook: A separate repo for first pomdp paper (AmNat)
Creative Commons Attribution 4.0 International
17 stars 4 forks source link

Small discrepancies between locally knitted pdf and published fig 2 #3

Open annakrystalli opened 4 years ago

annakrystalli commented 4 years ago

Hi!

I've been working on your paper as part of the Liverpool N8 CIR ReproHack. Thanks for submitting it! It's been really fun to work with.

I'm just finding small differences figure 2.

Here's the published plot

And here's what I'm getting when knitting the pdf:

Screenshot 2020-02-25 at 15 15 54
cboettig commented 4 years ago

Cool, thanks for reporting.

The core sarsop algorithm which underlies this (https://github.com/boettiger-lab/sarsop) is actually a stand-alone C++ program that does not integrate into R's random number generator (yes it's a form of optimization algorithm but isn't entirely deterministic). I believe a degree of variability in sarsop is expected. Of course, this should impact the POMDP solution only.

I'm actually not sure of the top of my head if seed is being locked for the simulations underneath all three scenarios (recall these results are the combination of a decision rule and then simulations of the underlying process). I may not enforced random seed at all, or perhaps appropriately across the parallel processing. In any event, the qualitative result should be robust to seed, so that at least is good to see!