search

MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.26k stars 21.43k forks source link

Snowflake warehouse #123270

Closed rwilliams-mpg closed 4 months ago

rwilliams-mpg commented 4 months ago

You should modify the warehouse settings in your documentation. Unless there is some overwhelming need, you should always start with an XS warehouse and always with a 1 minute auto suspend. I don't know much (anything) about Sentinel but I was asked to setup a connection for our security team. I don't know how often these will be run, but if a query was run once per hour on the settings suggested in the documentation, it would cost around $500/mo. With XS and 1 minute suspend you are closer to $50-75/mo.

Also, nothing is mentioned about a network policy. I don't know much about Azure as I use AWS but I assume since it is a function there is probably a way of binding it to an individual IP address and adding it to a network policy. A lot of snowflake environments will have network policies in place to restrict by IP so having some documentation about that would be helpful.

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

TPavanBalaji commented 4 months ago

@rwilliams-mpg Thanks for your feedback! We will investigate and update as appropriate.

rwilliams-mpg commented 4 months ago

Since I posted this, I was told by my security department that the default polling interval is 5m. Snowflake charges per credit for compute time. 1 credit on an Business Critical account is $4 and a Small warehouse uses 2 credits an hour, minimum 1 min billing each time the warehouse starts. Since the system polls every 5 minutes and shuts down the warehouse after 5 minutes of inactivity, it would end up running 24/7. Based on the recommended setup here it would end up being $5952/mo in costs.

$4/credit 2 credits/hour (Small) 24/hrs * 31

To put this in perspective, we run a snowflake instance running 40 + Tableau dashboards, dozens of automated reports, 3 Data engineers, ~20 analysts, hundreds of ETL's, 1000+ models in the warehouse and our total bill is less than half that a month.

cwatson-cat commented 4 months ago

@rwilliams-mpg Thanks for the feedback and information! I'll pass this along to the program managers that manage the data connector instructions in the upstream public repo: https://github.com/Azure/Azure-Sentinel/tree/master/Solutions/Snowflake. This content is autogenerated from the information about the data connector that's published there.

SaibabaBalapur-MSFT commented 4 months ago

@cwatson-cat Thank for quick responses. @rwilliams-mpg if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.

SaibabaBalapur-MSFT commented 4 months ago

@rwilliams-mpg We are going to close this thread as resolved but if there are any further questions regarding the documentation, please tag me in your reply and we will be happy to continue the conversation.