search

socialfoundations / whynot

A Python sandbox for decision making in dynamics
MIT License
418 stars 43 forks source link

Performative prediction on forecasting data [Feature request] #25

Open tmerrittsmith opened 3 years ago

tmerrittsmith commented 3 years ago

I'm working on a bunch of forecasting problems where performative prediction feedback loops are a challenge. I'd like to create a forecasting problem simulator in whynot. Any ideas of where I could start?

millerjohnp commented 3 years ago

Hi! There's a case study on performative prediction in the examples here that might be a good place to start. What sort of forecasting problems are you encountering? We've been on the lookout for more examples, and I'd be happy to include another performative prediction task if you have a concrete simulation/simulator in mind.

tmerrittsmith commented 3 years ago

We're working in demand forecasting using a range of models including auto-regressive models. When the forecasted demand underestimates the true demand, some percentage of the demand is not fulfilled, so if the forecast model is updated naively, it is updated using some modified demand which is less than the true demand thanks to previous underestimates from the forecast. I'm still trying to think of how to formulate this in a simulator, the difference with the example notebook is that for a forecast model the input variable is the historic y data, rather than having separate X and y distributions if that makes sense?

millerjohnp commented 3 years ago

Very interesting setting! One reference that might be of interest is Brown, Gavin, Shlomi Hod, and Iden Kalemaj. "Performative Prediction in a Stateful World." arXiv preprint arXiv:2011.03885 (2020)., which models a dynamic component like the one you're describing. It looks like that paper also includes some simulations here. If any of those are useful to you, I'm happy to discuss adding them to whynot.

tmerrittsmith commented 3 years ago

Thanks for the tips! I'll take a look and get back to you.

On Tue, Nov 2, 2021 at 10:25 PM John Miller @.***> wrote:

Very interesting setting! One reference that might be of interest is Brown, Gavin, Shlomi Hod, and Iden Kalemaj. "Performative Prediction in a Stateful World." arXiv preprint arXiv:2011.03885 (2020). https://arxiv.org/abs/2011.03885, which models a dynamic component like the one you're describing. It looks like that paper also includes some simulation here https://github.com/shlomihod/performative-prediction-stateful-world. If any of those are useful to you, happy to discuss adding them to whynot.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/zykls/whynot/issues/25#issuecomment-958179021, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC46UAS7KOGI3HUUKPGDOWTUKBQNLANCNFSM5GMGS4HA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.