A question about generalizing into domain specific models (e.g. time series)

[hesenp]

I noticed that we already have a few time series forecasting models as part of the examples. For these models we mostly write the models from ground up.

On the other side of the spectrum, there's all these open source packages like Prophet, Orbit which just rely on a few models internally, but have more of a peripheral functions to make them easy to use for folks who are not modeling experts. Coincidentally, Orbit has this LGT model which is highly similar to the SGT model that Numpyro has used as an example, linked above.

I wonder if there's any thoughts to building some domain specific models/packages based on numpyro? like numpyro-ts that uses numpyro as compute engine in the backend, but has utility functions in the front, so that people could do modeling easily?

an example code would like:

model = SGT() 
model.fit(data_frame)
model.predict(another_data_frame)

this could make the code / package very easy to get picked up by general practitioners.

[fehiepsi]

There are efforts in the past https://github.com/pyro-ppl/numpyro/pull/544 https://github.com/pyro-ppl/numpyro/pull/551 see e.g. the lynx example for having a forecast module similar to pyro.contrib.forecast. If anyone is interested, please feel free to reuse/modify the code there but I think it is better to start with some popular algorithms e.g. those in Rob Hyndman's forecasting book, which are already strong baselines.

[hesenp]

oh interesting. what's your thought on these code in the forecast folder @fehiepsi ? do you have plans to check them in some time?

[fehiepsi]

Hi @hesenp, I don't have a plan for it. Currently, for time series problems, I just construct the model from scratch. But I believe having a module for basic forecasting models would be useful for many people.

[hesenp]

I see. Please let me know if this ever crossed your mind @fehiepsi ?

I have this hunch that for many state space time series models, we can probably re-use a pattern like this:

class TimeSeriesBaseModelMixin() 
   """includes house keeping code that helps with sampling etc. """
  def fit(self, data): 
       # including a `scan` function that will use a `transition` function. 

class ARModel(TimeSeriesBaseModelMixin):
  def __init__(self, ... ): 
      # define the `transition` function which will be used by `scan` function 

of above, TimeSeriesBaseModelMixin will manage all the sampling and house keeping tasks. Within each model-specific class, we define transition function which will be used in the scan call. Using this pattern, we could probably scale ts models to things like ARIMA, and most state space models super easily.

Can't promise when i can get to this. but it seems like an interesting pattern that would show the power of numpyro when it happens.

[fehiepsi]

That API sounds reasonable to me. We can have default methods to get init values/ to-be-scanned values for traditional models while still provides extensions for some derived models. Like to include global terms as in Orbit

[hesenp]

Yeah Orbit is where I started to get the idea. The pattern is even more apparent in their StanEstimator. Basically the core model is specified through a bunch of stan files. The rest is all wrappers to massage data and run the estimation. Prophet is similar too. Prophet is basically all resolving around just one single stan file. The rest is all house keeping code.

[fehiepsi]

If you want, we can write a design doc to exercise models that we want to put into (personally, I prefer textbook models to complicated ones so it might be easier to find common patterns for my usage case). I'm happy to collaborate. :)

[hesenp]

Interesting ... seems Fritz and Martin gave some thought into this way back as well, as documented in this doc. I guess I'll just carve out some time in the weekend to doodle a prototype out.

[martinjankowiak]

that design doc is what lead to pyro.contrib.forecast

[fehiepsi]

@tomwallis designed an api for object-oriented wrappers around numpyro in https://github.com/ag-perception-wallis-lab/numpyro-oop I think that is the way to go for domain-specific models.