Open eromoe opened 3 years ago
Hi @eromoe, can you try inverting your mask:
- mask = torch.isnan(test_data)
+ mask = ~torch.isnan(test_data)
In Pyro's mask semantics, True means observed and False means missing.
@fritzo
Oh, I see, prefix_condition
is used for forecaster . Here is another problem come
here:
self.loc.shape
Out[12]: torch.Size([100, 44, 103, 657, 1])
batch_shape
Out[13]: torch.Size([100, 44, 103, 671, 1])
I have seen time length in prediction
= time length in zero_data
+ time length in covariance
.
So this expand would fail
noise_dist = noise_dist.expand(
noise_dist.batch_shape[:-2] + prediction.shape[-2:])
As to MaskedDistribution.expand
, mask size also doesn't fit to prediction size too (mask was fixed after model init )
def expand(self, batch_shape, _instance=None):
new = self._get_checked_instance(MaskedDistribution, _instance)
batch_shape = torch.Size(batch_shape)
new.base_dist = self.base_dist.expand(batch_shape) # exception at here
new._mask = self._mask # but here mask size is old too
if isinstance(new._mask, torch.Tensor):
new._mask = new._mask.expand(batch_shape)
super(MaskedDistribution, new).__init__(batch_shape, self.event_shape, validate_args=False)
new._validate_args = self._validate_args
return new
I also have a question about how pyro make prediction , I don't know why it need whole time length 671 here . For example . there are usual two kinds of timeseries forecast model,
But I can't figure out how pyro do here , there is a foreward method , but I don't understand how it make prediction . Could you make some explanation or provide some resouces to make this clear ?
@eromoe could you paste a little of your model code that triggers the above exception? Ideally could you provide a minimal model and fake data that triggers the error? It's hard for me to suggest a fix without being able to reproduce the error.
I also have a question about how pyro make prediction
pyro.contrib.forecast
does batch multi-step prediction (2). To predict you create a Forecaster
instance and call it on covariates longer than your data:
model = MyForeacastingModel(...)
forecaster = Forecaster(model, data, covariates_as_long_as_data) # fits a model
prediction = forecaster(data, covariates_that_are_longer_than_data, num_samples=100) # predicts
If you want to do windowed prediction then you'll need to create multiple Forecaster
instances, one per window that is a truncated form of data
, covariates
.
Sure, it is very easy to generate .
@fritzo Is there any thing I can help ? I am aslo trying to understand the source code , if you can provide some reading suggestion would be very nice : )
Hi @eromoe, I'll try to debug it this weekend. Thanks for providing a reproducible example!
Hi @eromoe, I managed to get your model working with a small fix (see below). The problem is that the mask
should have shape broadcastable to zero_data.shape
. But your self.mask
is stored once in .__init__()
whereas zero_data
changes depending on whether the model is being trained (which works in your case) or being used for prediction (which fails in your case because the expanded size of zero_data
no longer matches the static size of self.mask
).
My proposed solution is to apply the mask only during training. This works because masking only affects the log_prob
and log_prob
is used only during training, not during prediction. We can distinguish training from prediction because the prediction
tensor requires gradients only during training (the prediction method forecaster.__call__()
is called in a torch.no_grad()
context). To apply this solution you can simply change
- if self.mask is not None:
+ if self.mask is not None and prediction.requires_grad:
noise_dist = noise_dist.mask(self.mask)
I admit this is kind of a hack; if you find a more elegant solution please share on this thread. If you're feeling ambitious, we'd welcome an additional section in one of the tutorials demonstrating how to do masking 😄
Here's the full script:
Thank you for the detailed explanation ! I would like to help the new section , just need some times 😄
Missing observation is very common in real work , but I doesn't found a way to write a correct model . And tutorial http://pyro.ai/examples/forecasting_iii.html do not cover this problem : nan in real data . For example : in 2010, there was only 48 station in 2011, 3 station was closed and 5 new opened, 50 stations . in 2012, 2 station was closed in 2011 reponed …
For another example : My data is salecount of various products in many stores . I have reshape the szie to torch.Size([44, 103, 671, 1]) , means: 44 stores, 103 products , 671 days salecount . Some stores may be closed in different days, so as products would be off.shelf by many reason .
random 10 products history salecount :
Create matrix must have nan values , and
According to https://forum.pyro.ai/t/how-to-ignore-nan-values-when-do-hierachical-forecast/2412
I alter forecast_iii code to below, structure is totally same , I just change some names .
I add a mask to noise distribution , but without luck , it doesn't work in prediction .