Kusto query error in Log Analytics Workbook for Synapse Spark

[robert4os]

Under the section 'Use the sample workbook to visualize the metrics and logs', there is a download of a LogAnalytics Workbook ('Azure_Synapse_Spark_Application.workbook') which contains a kusto query...

This kusto query references a column 'exception_stacktrace_s' which does not exist and the query fails in the portal:

The problematic query in the workbook is:

"query": "SparkLoggingEvent_CL\r\n| where workspaceName_s == \"{Workspace}\" and clusterName_s == \"{SparkPool}\" and livyId_s == \"{LivyId}\"\r\n| project TimeGenerated, executorId_s, Level, logger_name_s, Message, exception_stacktrace_s\r\n| where iff(\"{Logger:label}\" == \"All\", true, logger_name_s in ({Logger}))\r\n| where iff(\"{Level:label}\" == \"All\", true, Level in ({Level}))\r\n| order by TimeGenerated asc\r\n| limit 1000", <<

The following is my suggestion for a rewrite of the query that works and allows for the absence or presence of the column:

"query": "SparkLoggingEvent_CL\r\n| where workspaceName_s == \"{Workspace}\" and clusterName_s == \"{SparkPool}\" and livyId_s == \"{LivyId}\"\r\n| project TimeGenerated, executorId_s, Level, logger_name_s, Message, exception_stacktrace_s=column_ifexists('exception_stacktrace_s', '')\r\n| where iff(\"{Logger:label}\" == \"All\", true, logger_name_s in ({Logger}))\r\n| where iff(\"{Level:label}\" == \"All\", true, Level in ({Level}))\r\n| order by TimeGenerated asc\r\n| limit 1000", <<

Background: I assume that when errors occur and get logged the column 'exception_stacktrace_s' would be populated and exist, however, in my environment this did not happen hence the error. The query rewrite can handle the optional existence of that column.

best, Robert

[Enter feedback here]

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: f9b48422-56c8-1928-97d9-69cb8ee21b18
• Version Independent ID: fcdbb080-daa1-bac6-3cba-8ab912e8f500
• Content: Monitor Apache Spark applications with Azure Log Analytics - Azure Synapse Analytics
• Content Source: articles/synapse-analytics/spark/apache-spark-azure-log-analytics.md
• Service: synapse-analytics
• Sub-service: spark
• GitHub Login: @Jejiang
• Microsoft Alias: jejiang

[RyanHill-MSFT]

Thanks for the feedback! We are currently investigating and will update you shortly.

[ShaikMaheer-MSFT]

@Jejiang Could you please validate customer feedback and update query as appropriate? Thank you.

[bandersmsft]

Thanks for your dedication to our documentation. Unfortunately, at this time we have been unable to review your issue in a timely manner and we sincerely apologize for the delayed response. We are closing this issue for now, but if you feel that it's still a concern, please respond and let us know. If you determine another possible update to our documentation, please don't hesitate to reach out again. #please-close