spark-monitor branch l4jv2 to support custom logging after DBR version upgrade to 12.2 are having column mapping issues. Some of the SparkLoggingEvent_CL columns are wrongly mapped. I am not sure why the below new columns are introduced in the new version which supposed to be mapped to the existing columns.
10.4 vs 11.0 and above mappings list.
Level vs log_level_s, thread_name_s vs process_thread_name_s, logger_name_s vs log_logger_s, applicationName_s vs sparkAppName_s, nodeType_s vs sparkNode_s
Below columns are having the similar values in both version but mapped to different column in 11.3 and 12.2
spark-monitor branch l4jv2 to support custom logging after DBR version upgrade to 12.2 are having column mapping issues. Some of the SparkLoggingEvent_CL columns are wrongly mapped. I am not sure why the below new columns are introduced in the new version which supposed to be mapped to the existing columns.
10.4 vs 11.0 and above mappings list. Level vs log_level_s, thread_name_s vs process_thread_name_s, logger_name_s vs log_logger_s, applicationName_s vs sparkAppName_s, nodeType_s vs sparkNode_s
Below columns are having the similar values in both version but mapped to different column in 11.3 and 12.2
<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">
main branch | l4jv2 | -- | -- | -- DBR 10.4 | DBR 12.2 | Example Values Level | log_level_s | INFO thread_name_s | process_thread_name_s | Thread-66 logger_name_s | log_logger_s | log info applicationName_s | sparkAppName_s | Databricks Shell nodeType_s | sparkNode_s | driver