n_iter in monte_carlo.py always set to T

[shenkev]

Sorry in advance if I have a conceptual misunderstanding but I'm wondering why in monte_carlo.py the variable n_iter is being set to the number of empirical samples (call this T) in the posterior. Relevant line: kwargs['n_iter'] = np.amin([qz.params.shape.as_list()[0]

From the docs, n_iter is "if controlling inference manually, it is the expected number of calls to update()". Are we getting only 1 monte carlo sample per update() call? Do we need to worry about burn-in here? Shouldn't n_iter be greater than number of empirical samples in this case?

The doc says:

No warm-up is implemented. Users must run MCMC for a long period of time, then manually burn in the Empirical random variable.

how would you do this?

I tried calling inference.update() for x > T number of times

for i in range(x):
    info_dict = inference.update()

and I get an out of index error (here T=1000, x=1001).

InvalidArgumentError (see above for traceback): indices = 1000 is not in [0, 1000)
     [[Node: ScatterUpdate_1 = ScatterUpdate[T=DT_FLOAT, Tindices=DT_INT32, _class=["loc:@Variable_11"], use_locking=true, _device="/job:localhost/replica:0/task:0/cpu:0"](Variable_11, iteration_1/read, cond_1/Merge)]]

I think this suggests monte_carlo.py class is initialized to expect T calls to update() and calling update() more than T times messes with the implementation? How would you burn in/keep every m samples?

Relevant code in monte_carlo.py:

  def initialize(self, *args, **kwargs):
    kwargs['n_iter'] = np.amin([qz.params.shape.as_list()[0] for
                                qz in six.itervalues(self.latent_vars)])
    super(MonteCarlo, self).initialize(*args, **kwargs)

    self.n_accept = tf.Variable(0, trainable=False, name="n_accept")
    self.n_accept_over_t = self.n_accept / self.t
    self.train = self.build_update()

    self.reset.append(tf.variables_initializer([self.n_accept]))

    if self.logging:
      tf.summary.scalar("n_accept", self.n_accept,
                        collections=[self._summary_key])
      self.summarize = tf.summary.merge_all(key=self._summary_key)

[dustinvtran]

Are we getting only 1 monte carlo sample per update() call? Do we need to worry about burn-in here? Shouldn't n_iter be greater than number of empirical samples in this case?

Yes. Yes. No. Burn-in is just an argument to throw away initial samples at the start of the Markov chain. In practice, you set the number of Empirical samples to be, say, twice as large as you want and take only the second half. An MCMC example with burn-in is examples/bayesian_linear_regression.py.

[shenkev]

Ah I see, so to "take only the second half," we do something like:

qz_kept = qz.params[to_keep_index:]?

the _sample_n(self, n, seed=None) function doesn't seem to have an argument to keep only the later samples

edit: ah ok you can wrap it in a variable: Empirical(qz.params[to_keep_index:])