apache / superset

Apache Superset is a Data Visualization and Data Exploration Platform
https://superset.apache.org/
Apache License 2.0
61.94k stars 13.57k forks source link

Uncertainty intervals of forecast are wrong when it goes negative. #21734

Open Dapid opened 1 year ago

Dapid commented 1 year ago

Uncertainty intervals of forecast never goes negative.

How to reproduce the bug

1 Take any data that includes negative values. 2 Run a forecast, including uncertainty interval

Expected results

I'd expect the uncertainty to be around the forecast.

Actual results

The uncertainty interval is clipped from below at 0, and behaves oddly.

Screenshots

Screenshot 2022-10-07 at 12-48-06 forecast

Environment

(please complete the following information):

Checklist

Make sure to follow these steps before submitting your issue - thank you!

Additional context

Here is some data one can use. I used Date as the X axis and min(temperature) for Y (days, weeks, or months, work fine). Note that using max(temperature) instead yields nice-looking results, since almost all of it is positive.

ShaliniIruvuru commented 1 year ago

Hi it will be very help for us to resolve this issue if you provide detailed information,

rusackas commented 7 months ago

I'm not sure if this is still an issue in current versions of Superset (3.x). If it is, we can re-open this, or feel free to open a new issue with updated context and a reproducible case using example data. We're no longer supporting Superset 2.x or prior, and it's been a while since this thread saw any activity, so I'm closing this as stale.

Dapid commented 7 months ago

The problem is still here in 3.1. You need any data with negative values.

For example, creating a line chart based on this query, and adding a forecast works as expected:

SELECT DATETIME(year, 'start of day') AS year,
       MIN(SP_POP_AG14_MA_IN) AS "MIN(SP_POP_AG14_MA_IN)"
FROM main.wb_health_population
GROUP BY DATETIME(year, 'start of day')
ORDER BY "MIN(SP_POP_AG14_MA_IN)" DESC
LIMIT 10000
OFFSET 0;

image

Changing the sign of the value yields instead:

image

And in fact, plotting MIN(SP_POP_AG14_MA_IN) - 170 works fine:

image

But plotting MIN(SP_POP_AG14_MA_IN) - 172 is odd again, even though there aren't any negative values.

image

rusackas commented 7 months ago

Ahh, thanks for the context! Re-opening!

rusackas commented 2 months ago

This issue has gone stale, but I'm hesitant to close it. Not sure if @villebro took a look at it and/or has any clues to go on.

Dapid commented 2 months ago

The problem remains on v4.0.2.

image

I can't find where the plot is actually being done, so I haven' been able to track down the problem further. But I have checked the raw data, and prophet is modelling it correctly, is just the display.

The issue doesn't have much activity, but the bug persists, so I think closing it is wrong.