Decision process for moving forecasts from research to operation

[jpeters7]

This project came from an EFI Translation working group call on June 13, 2023. While it is most likely too late to consider this project for the Unconference, I wanted to include it here since this repo of project ideas will live on past the Unconference.

Chris Brown (U of Maryland) was a major contributor to this idea during the call.

Question: How do we (society, funding agencies, research groups, private industry, etc) decide which ecological forecasts should be transitioned to operations?

What is the decision process for having forecasts move forward from research to a sustainable, reliable forecast that goes to the user? There are a lot of forecasts that are created, but not all of them are qualified to be made operational. Two examples of criteria brainstormed during the Translation working group call include 1) funding (there is limited funding so how to decide what gets funded) and 2) the need to identify potential users.

Task:

• Develop a list of criteria required for a forecast to be selected for operations
• Get initial input for this list from a range of groups involved with the forecast process (forecast creators, forecasts users, funding sources)
• Get iterative feedback from the different groups about the list of criteria and set up a group discussion about priorities

[trashbirdecology]

This is very much related to the proposal being led by the CI/Methods WG (best POC @jzwart). So anyone interested in this topic may be interested in one or more of the proposed workshop sessions.

[jpeters7]

Yes! The proposal had come up in this discussion, so it is great to hear you also see the connection @trashbirdecology. @jzwart, I'm pinging you as well to add you to the conversation.

[ARSjodin]

This semi fits into some things I've been thinking about in regards to what makes a "good" forecast. I think whether or not a forecast becomes operational depends on how "good" it is. Of course, what does it mean to be a good forecast? I don't necessarily have answers, but I've been thinking about it from the perspective of how the makeup of the forecasting team could influence the success of a forecast. This black thought hole was precipitated by a talk Owen Petchey gave during one of the European chapter's zoom presentations and spurred on by some of the work shared in Gardner and Tetlock's Superforecasting book.

[jzwart]

cool, nice to see this come up in other working groups and potentially discussed at the unconference, and I'll keep tabs on this discussion as it moves forward. Another group to keep in mind on this topic that is a mixture of academics and agencies is the Cooperative Institute for Research to Operations in Hydrology https://ciroh.ua.edu/

[noamross]

I'd be very interested in spending some time on this, as well. Maybe we can do a short session or lunch discussion on it.

[melissakenney]

Michael Gerst and I have worked on this from a forecast usability perspective and operationalizing the social science insights to make forecasts usable. I think it's useful to see this as part of the process. And I'm the UMN lead of CIROH and we can have a separate conversation about that...

[ashander]

I'd be very interested in spending some time on this, as well. Maybe we can so a short session or lunch discussion on it.

+1 to this