Explore superset datasource capabilities and limitations

[michaelkohn]

We'd like to get a better understanding of datasources in Superset

• What are the options/strategies/best practices for datasource refresh?
• What are the security capabilities of datasources? Do we have to give users access to specific datasources?
• Is there a timeout limit for datasources? For example... what happens if the query takes 5 hours to complete?
• Is there a maximum size for datasources? For example... what if the datasource brings back 500 million rows or the results set is 20GB?

[Pall-v]

Some high-level answers with references:

Dashboard refresh:

• Can be different for each chart on the dashboard and they can be staggered: https://superset.incubator.apache.org/faq.html#how-to-limit-the-timed-refresh-on-a-dashboard

What are the security capabilities of datasources? Do we have to give users access to specific datasources?

• Access can be restricted by DB and/or datasource. Access to specific datasets can be managed via creating additional users roles: https://superset.incubator.apache.org/security.html?highlight=security#managing-gamma-per-data-source-access

Is there a timeout limit for datasources? For example... what happens if the query takes 5 hours to complete?

• The default timeout is 6 hours. it is configurable: https://superset.incubator.apache.org/faq.html?highlight=timeout#why-are-my-queries-timing-out

Is there a maximum size for datasources? For example... what if the datasource brings back 500 million rows or the results set is 20GB?

• "It can be gigantic!": https://superset.incubator.apache.org/faq.html#how-big-can-my-data-source-be