Open brijos opened 1 year ago
Both persona would require data for a longer period, ideally a year. With less than a year of data, you are likely to miss the season variations which might not just depend on last 90 days or 120 days of trend. Ideally there should be a base model to start and each of the persona supplement the data they have to derive their needs.
As we're working toward introducing Forecasting, I'd like to share some of the exploratory work for the DevOps metrics use case from the UX perspective.
“As someone who is responsible for critical services, I want to look at my dashboard visualizations and forecast where metrics are moving, so I can understand if I can take proactive actions and change the outcome”.
"I'm aware of a data point that is not included in the model (seasonal trend longer than 90-120 days, planned external event) to forecast the trend. I would like to see how it will affect the trend."
From an existing visualization
Visualize your data and forecast
Create alerting monitor and augment it with a predictive alert trigger
Stay tuned for more updates, and please share your feedback on any aspects of this proposal. I also have some open questions listed below:
Please share any additional feedback!
Both persona would require data for a longer period, ideally a year.
I'm not sure I agree with this statement or the postulation in the issue itself (e.g., the persona will need specific data windows). While most people in either of these personas will likely say more data is better (not true in many cases IMO), the time window needed to meaningfully forecast will vary fairly significantly based on the application, use case, data variability, etc. In particular, the ability to create a reasonable range of values in which future values of interest are likely to fall has many factors and data over time (i.e., as measured in length over time) is only one of those.
For example, if I am charged with monitoring calls to an API, it might be very feasible to alert on a forecasted utilization rate if the API calls have had low variability and high usage in the last week and a very clear trend in upward utilization in the past day. This is one of hundreds of examples where I would absolutely not need a year of data to meaningful forecast values.
@brijos Is there a reason we need to specify the required data retention or the length of time each personal "needs" data given it is so use case specific? I'm not sure how this informs a technical requirement in a world were we would want to allow users to select a time range that is suitable to their use case (and each user will want to make their own tradeoff decision between cost, speed, accuracy, etc.).
forecast where metrics are moving
Minor nit here. I think it's more accurate to say "I want to see where metrics are forecasted to be" as the entire feature is to remove the need for the user to employ naive, intuitive, or heuristic forecasting.
seasonal trend longer than 90-120 days
I'm not sure I understand this from a forecasting or analytics use case as written. Traditionally, seasonality is a characteristic of data over a time period where there is some regular and predictable changes that recur. Are we trying to say that the user can simply change the time range of the input data to allow for different windows of time for scenario forecasting?
For example, if I was a user and I was performing a forecast, I want the model to look for a cycle of seasonality based on the data, e.g., 12-month cycle of seasonality for monthly data, four-quarter cycles for quarterly data, weekly seasonality for daily data, hourly seasonality for intraday data, etc. OR based on cycles of seasonality I choose in my scenario planning/exploration. Is this what we want to support long-term?
What kind of configurations you’d like to have for forecasting your metrics?
So, I can personally think of a few that I'd want to see (depending on the underlying configurability and capabilities). These would include;
UX perspective
One other callout that may be worthwhile to think about: The same UX treatments that are beneficial to forecasts would be useful in other analysis such as trend line or line of best fit. There are a ton of predictive modeling functions outside of forecasting that could be / will be added in the future that might be worth thinking about.
What are you proposing?
The current log analytics solution is great at showing how things are performing in the moment. When looking at log metrics, users also want to understand where things are trending so that they can plan accordingly. Forecasting will show the user where their metrics are headed so that they can prepare accordingly (e.g. disk space, server instance upgrades, or even license usage if tracked).
Show a future view of time series metrics displayed on Dashboards which will feed into Observability and Security Analytics
Forecast when something may trigger an alert in the future to help users stay on top of their areas of responsibility
Create a workflow which allows someone to upload their dataset and validate that it is correct ahead of performing the forecasting job
Include “what if” scenario support which allow the user to work through scenarios like what if traffic increased 4x what would happen to my systems
Have popular forecasting dashboard templates to make it easy to setup
Easily share a forecasting dashboards with colleagues and the community
Tee up a scheduled report of a specific dashboard to be reviewed quarterly
What problems are you trying to solve?
There are two personas that we propose to solve for. The first priority is the developer operations persona who wants to view dashboards and understand where metrics are headed and the second priority is the business analyst persona. A key differentiator between the personas would be data size. The developer operations persona will likely only have 90-120 days worth of data available to them given that saving application logs is costly. The developer operations persona wants to make sure that the applications, service tiers, or services are up and fully functional. They live more in the moment and would like to be proactively notified if their area of responsibility is going to have problems.
The second persona is the business analyst persona who wants to look at data over a longer period of time (120+ days). The data the business analyst is looking at has seasonality trends in it. The business analyst wants to understand how to project out financial metrics such as revenue or inventory levels.
User Stories
Developer Operations Persona (90 - 120 days of data)
Business Analyst Persona (120 days + of data)
Reporting
Administration
Outstanding Questions