MarileeTurscak-MSFT
commented
2 years ago @jasonhorner Thanks for your feedback! We will investigate and update as appropriate.
Closed jasonhorner closed 4 months ago
MarileeTurscak-MSFT
commented
2 years ago @jasonhorner Thanks for your feedback! We will investigate and update as appropriate.
bandersmsft
commented
4 months ago Thanks for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. We are closing this issue for now, but if you feel that it's still a concern, please respond and let us know. If you determine another possible update to our documentation, please don't hesitate to reach out again. #please-close
I've been struggling with some performance issues with serverless against relatively small files (10MB - 2GB) parquet files.
There doesn't seem to be a way to get statistics or information about a query plan so troubleshooting performance is a painful trial and error approach.
some additional topics that might be useful to speak to in this section: https://docs.microsoft.com/en-us/azure/synapse-analytics/sql/resources-self-help-sql-on-demand#performance
I've done some testing around these areas but results are inconsistent and not very repeatable.
I'm assuming most of the general MPP design guidance still applies; but there isn't a lot of detail about the internals of the Polaris query processor and the query execution itself that makes it difficult to tune a workload.
One feature enhancement that would help greatly is to include the number of files scanned, and if the same file was scanned multiple times (similar to statistics io feature in sql server) with query execution. This would allow to validate the effectiveness of partition elimination and better understand query patterns that are leading to sub optimal I/O.
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.