Critical Error with Updated Dependencies

[jbhatch]

Using updated versions of torch, torchvision, and pyro dependencies, an error (below) occurs in the instance of SVI where the event_dims between the model and guide disagree at site 'module$$$out.weight': 0 vs 1. Additionally, .independent() is deprecated and it is recommended that it be changed its replacement, .to_event(), in the following line of the guide: " outw_prior = Normal(loc=outw_mu_param, scale=outw_sigma_param).independent(1)". Unfortunately, changing from .independent(1) to .to_event(1) in the guide does not rectify the event_dims mismatch error.

Please help. I would very much like to be able to use your Bayesian Neural Network script as it deals with rejecting untrained classes in the test data according to probability.

P.S. On a side note regarding a previously discussed issue, the MNIST data is inaccessible using torchvision's datasets.MNIST as used in bnn.ipynb. I checked other image datasets such as Fashion-MNIST, and that was readily available. Currently, the MNIST data can be obtained with this command: "!wget www.di.ens.fr/~lelarge/MNIST.tar.gz".

Error Messages: /usr/local/lib/python3.7/dist-packages/pyro/primitives.py:451: FutureWarning: The random_module primitive is deprecated, and will be removed in a future release. Use pyro.nn.Module to create Bayesian modules from torch.nn.Module instances. "modules from torch.nn.Module instances.", FutureWarning)

ValueError Traceback (most recent call last)

in () 6 for batch_id, data in enumerate(train_loader): 7 # calculate the loss and take a gradient step ----> 8 loss += svi.step(data[0].view(-1,28*28), data[1]) 9 normalizer_train = len(train_loader.dataset) 10 total_epoch_loss_train = loss / normalizer_train /usr/local/lib/python3.7/dist-packages/pyro/infer/svi.py in step(self, *args, **kwargs) 126 # get loss and compute gradients 127 with poutine.trace(param_only=True) as param_capture: --> 128 loss = self.loss_and_grads(self.model, self.guide, *args, **kwargs) 129 130 params = set(site["value"].unconstrained() /usr/local/lib/python3.7/dist-packages/pyro/infer/trace_elbo.py in loss_and_grads(self, model, guide, *args, **kwargs) 129 loss = 0.0 130 # grab a trace from the generator --> 131 for model_trace, guide_trace in self._get_traces(model, guide, args, kwargs): 132 loss_particle, surrogate_loss_particle = self._differentiable_loss_particle(model_trace, guide_trace) 133 loss += loss_particle / self.num_particles /usr/local/lib/python3.7/dist-packages/pyro/infer/elbo.py in _get_traces(self, model, guide, args, kwargs) 168 else: 169 for i in range(self.num_particles): --> 170 yield self._get_trace(model, guide, args, kwargs) /usr/local/lib/python3.7/dist-packages/pyro/infer/trace_elbo.py in _get_trace(self, model, guide, args, kwargs) 56 """ 57 model_trace, guide_trace = get_importance_trace( ---> 58 "flat", self.max_plate_nesting, model, guide, args, kwargs) 59 if is_validation_enabled(): 60 check_if_enumerated(guide_trace) /usr/local/lib/python3.7/dist-packages/pyro/infer/enum.py in get_importance_trace(graph_type, max_plate_nesting, model, guide, args, kwargs, detach) 48 graph_type=graph_type).get_trace(*args, **kwargs) 49 if is_validation_enabled(): ---> 50 check_model_guide_match(model_trace, guide_trace, max_plate_nesting) 51 52 guide_trace = prune_subsample_sites(guide_trace) /usr/local/lib/python3.7/dist-packages/pyro/util.py in check_model_guide_match(model_trace, guide_trace, max_plate_nesting) 252 if model_site["fn"].event_dim != guide_site["fn"].event_dim: 253 raise ValueError("Model and guide event_dims disagree at site '{}': {} vs {}".format( --> 254 name, model_site["fn"].event_dim, guide_site["fn"].event_dim)) 255 256 if hasattr(model_site["fn"], "shape") and hasattr(guide_site["fn"], "shape"): ValueError: Model and guide event_dims disagree at site 'module$$$out.weight': 0 vs 1

[paraschopra]

Sorry, I'm no longer actively maintaining this. If you have a fix, please send.

[tgeller08]

I encountered a similar issue and am working on a fix. For now, a temporary solution is to install an earlier version of pyro in the notebook or virtual environment. For example, run !pip3 install pyro-ppl==1.4.0 instead.

[mehmetavnicelik]

I encountered a similar issue and am working on a fix. For now, a temporary solution is to install an earlier version of pyro in the notebook or virtual environment. For example, run !pip3 install pyro-ppl==1.4.0 instead.

I tried this but got importing error, probably because of the version is deprecated. did you find any other solution? thanks in advance

[Sitheral]

I've figured out a fix for this. You might try modifying the model and guide function in the following manner def model(x_data, y_data):

fc1w_prior = Normal(loc=torch.zeros_like(net.fc1.weight), scale=torch.ones_like(net.fc1.weight)).to_event(2)
fc1b_prior = Normal(loc=torch.zeros_like(net.fc1.bias), scale=torch.ones_like(net.fc1.bias)).to_event(1)

outw_prior = Normal(loc=torch.zeros_like(net.out.weight), scale=torch.ones_like(net.out.weight)).to_event(2)
outb_prior = Normal(loc=torch.zeros_like(net.out.bias), scale=torch.ones_like(net.out.bias)).to_event(1)

priors = {'fc1.weight': fc1w_prior, 'fc1.bias': fc1b_prior,  'out.weight': outw_prior, 'out.bias': outb_prior}
# lift module parameters to random variables sampled from the priors
lifted_module = pyro.random_module("module", net, priors)
# sample a regressor (which also samples w and b)
lifted_reg_model = lifted_module()

lhat = log_softmax(lifted_reg_model(x_data))

pyro.sample("obs", Categorical(logits=lhat).to_event(1), obs=y_data)

def guide(x_data, y_data):

# First layer weight distribution priors
fc1w_mu = torch.randn_like(net.fc1.weight)
fc1w_sigma = torch.randn_like(net.fc1.weight)
fc1w_mu_param = pyro.param("fc1w_mu", fc1w_mu)
fc1w_sigma_param = softplus(pyro.param("fc1w_sigma", fc1w_sigma))
fc1w_prior = Normal(loc=fc1w_mu_param, scale=fc1w_sigma_param)
# First layer bias distribution priors
fc1b_mu = torch.randn_like(net.fc1.bias)
fc1b_sigma = torch.randn_like(net.fc1.bias)
fc1b_mu_param = pyro.param("fc1b_mu", fc1b_mu)
fc1b_sigma_param = softplus(pyro.param("fc1b_sigma", fc1b_sigma))
fc1b_prior = Normal(loc=fc1b_mu_param, scale=fc1b_sigma_param)
# Output layer weight distribution priors
outw_mu = torch.randn_like(net.out.weight)
outw_sigma = torch.randn_like(net.out.weight)
outw_mu_param = pyro.param("outw_mu", outw_mu)
outw_sigma_param = softplus(pyro.param("outw_sigma", outw_sigma))
outw_prior = Normal(loc=outw_mu_param, scale=outw_sigma_param)
# Output layer bias distribution priors
outb_mu = torch.randn_like(net.out.bias)
outb_sigma = torch.randn_like(net.out.bias)
outb_mu_param = pyro.param("outb_mu", outb_mu)
outb_sigma_param = softplus(pyro.param("outb_sigma", outb_sigma))
outb_prior = Normal(loc=outb_mu_param, scale=outb_sigma_param)
priors = {'fc1.weight': fc1w_prior.to_event(2), 'fc1.bias': fc1b_prior.to_event(1), 'out.weight': outw_prior.to_event(2), 'out.bias': outb_prior.to_event(1)}

lifted_module = pyro.random_module("module", net, priors)

return lifted_module()