Neural Net minibatch code not working

[sachinruk]

Hi all,

I am trying to run the code in http://pymc-devs.github.io/pymc3/notebooks/bayesian_neural_network_advi.html . However, I am running into the error: TypeError: 'list' object is not an iterator when I run the block:

with neural_network:
    # Run advi_minibatch
    v_params = pm.variational.advi_minibatch(
        n=50000, minibatch_tensors=minibatch_tensors,
        minibatch_RVs=minibatch_RVs, minibatches=minibatches,
        total_size=total_size, learning_rate=1e-2, epsilon=1.0
    )

Thomas suggested that it may be because I am running a legacy version of Python (as shown in the comments in: http://twiecki.github.io/blog/2016/06/01/bayesian-deep-learning/#comment-2752623456). However, I am running the anacondas Python 3.5 version. Normal VB using v_params = pm.variational.advi(n=50000) works perfectly fine.

The full error is posted below:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-25-2df3376dab1d> in <module>()
     12         n=50000, minibatch_tensors=minibatch_tensors,
     13         minibatch_RVs=minibatch_RVs, minibatches=minibatches,
---> 14         total_size=total_size, learning_rate=1e-2, epsilon=1.0
     15     )

/Users/sachin/anaconda/lib/python3.5/site-packages/pymc3-3.0-py3.5.egg/pymc3/variational/advi.py in advi_minibatch(vars, start, model, n, n_mcsamples, minibatch_RVs, minibatch_tensors, minibatches, total_size, learning_rate, epsilon, random_seed, verbose)
    178     elbos = np.empty(n)
    179     for i in range(n):
--> 180         uw_i, g, e = f(*next(minibatches))
    181         elbos[i] = e
    182         if verbose and not i % (n//10):

TypeError: 'list' object is not an iterator

Note 1. Just so you know I do not have an izip function in itertools as was suggested by Tom in his blog comments.

Any thoughts? Thank in advance.

[jordangumm]

The expected minibatches parameter now expects a generator of iterables instead of the iterable of generators the tutorial provides. Could you try and change the create_minibatch function and minibatches variable to:

def create_minibatch(data):
    rng = np.random.RandomState(0)
    while True:
        ixs = rng.randint(len(data), size=50)
        yield data[ixs]

minibatches = zip(
    create_minibatch(X_train),
    create_minibatch(Y_train),
)

[sachinruk]

that did the trick!