search

pymc-devs / pymc2

THIS IS THE **OLD** PYMC PROJECT (VERSION 2). PLEASE USE PYMC INSTEAD:
http://pymc-devs.github.com/pymc/
Other
879 stars 229 forks source link

Modifying chains and calculating DIC? #111

Closed sammosummo closed 8 years ago

sammosummo commented 8 years ago

I recently ran some models and found my burn-in to be insufficient. Is it possible to remove samples from my chains and then re-calculate DIC? The models took about a week, so it's not convenient to re-run them.

fonnesbeck commented 8 years ago

Which back end are you using? The DIC calculation uses the model's db.trace(), so you could go in and truncate the chains manually, via something like:

model._trace[chain] = model._trace[chain][burn:]
sammosummo commented 8 years ago

I'm using the pickle backend.

This seems to work (at least, it produces slightly different DIC values after removing some of the leading samples). For the record, I used something like this:

stochastics = m.get_stochastics()

for node in stochastics.node:

        node.trace._trace[0] = node.trace._trace[0][burn:]

Where m is my model and burn is the number of leading samples removed. Does this match up with what you were suggesting?

fonnesbeck commented 8 years ago

Yup. Glad it worked. Doing this sort of thing will be much easier in PyMC3.

rdp-dn commented 8 years ago

The _trace member is for ram and pickle databases, right? How would you do this with an hdf5 database? Also, if my model has deterministics, can I truncate them the with the same method?

fonnesbeck commented 8 years ago

Deterministics work the same way; traces are the same regardless of node type.

The HDF5 backend is a little more complicated, so you don't have direct access to the trace values as you do with other backends. Have a peek at the pymc/database/hdf5.py source to see how PyMC interacts with HDF5.