search

MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.29k stars 21.47k forks source link

Concurrency in Azure Synapse SQL Serverless #106686

Closed xela-95 closed 1 year ago

xela-95 commented 1 year ago

Hi, I would like to know more about, concurrency limits in Azure Synapse SQL Serverless, since the only information I found are in this paragraph. However, I would like to know more in detail what are the max IOPS mentioned and how to recognize when throttling is occurring. Moreover, when one client is querying parquet data from the data lake house via SQL Serverless, is that file locked for other clients or not? I tried with multiple parallel queries and it seems to perform some sort of locking.

Thank you

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

SaibabaBalapur-MSFT commented 1 year ago

@xela-95 Thanks for your feedback! We will investigate and update as appropriate.

AjayBathini-MSFT commented 1 year ago

Hi @xela-95

Sure, I can help you with that. According to the documentation, Azure Synapse SQL Serverless tracks resource utilization by assigning concurrency slots to each query. The system puts queries into a queue based on importance and concurrency slots. Queries wait in the queue until enough concurrency slots are available. Importance and concurrency slots determine CPU prioritization.

Regarding the maximum IOPS, the information is not mentioned in the documentation. However, you can monitor the performance of your queries and identify if there is any throttling occurring. If you are experiencing performance issues, you can reach out to the Microsoft support team for further assistance. As for the file locking, it is not specified in the documentation. However, you can try running your queries in parallel and monitor the performance to see if there is any locking occurring. If you are experiencing performance issues, you can reach out to the Microsoft support team for further assistance.

I'd recommend working closer with our support team via an [Azure support request] (https://docs.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request).

rdknz commented 1 year ago

Hello - can you please point a piece of Documentation that talks about queue slots? This is only mentioned in the doco for Dedicated SQL pool, the question above is regarding Serververless.

https://learn.microsoft.com/en-us/azure/synapse-analytics/sql-data-warehouse/memory-concurrency-limits