databrickslabs / overwatch

Capture deep metrics on one or all assets within a Databricks workspace
Other
221 stars 60 forks source link

Ambiguous reference to the field `vendor_UNIQUESUFFIX_-820075192`. #1233

Closed msadlocha31 closed 3 weeks ago

msadlocha31 commented 4 weeks ago

We recently discovered the problem with our Production Overwatch and tried to fix that ourselves but so far our methods are not pushing it forwards. The problem is within the Bronze_Clusters_Snapshot that fails due to Ambiguous reference to the field vendor_UNIQUESUFFIX_-820075192.

We tried on our INT environment to upgrade the version of OVW maven repository that end up with the same error. The other solution we tried was to rollback all the related modules and run it again which also did not help. LOGS_PRD.md

mohanbaabu1996 commented 4 weeks ago

Hi, @msadlocha31 ,

We've identified this bug and root cause as well. We expect the new version to be released early next week. Please let me know which version currently you're using.

Thanks!

msadlocha31 commented 4 weeks ago

Hi @mohanbaabu1996,

On Int there is version 0.8.1.1 and the error its actually now bit different: "Bronze_Clusters_Snapshot: Validating Input Schemas Wrote 726862 bytes. FAILED: 1002-Bronze_Clusters_Snapshot Module: Failed

Failed to merge fields 'spec' and 'spec' com.databricks.sql.transaction.tahoe.DeltaAnalysisException: Failed to merge fields 'spec' and 'spec' at com.databricks.sql.transaction.tahoe.schema.SchemaMergingUtils$.$anonfun$mergeSchemas$1(SchemaMergingUtils.scala:207)"

On PROD we stay with old 0.7.2.2.1

mohanbaabu1996 commented 4 weeks ago

Thanks, Have you started running Overwatch from 0811 or updated to 0811 from the previous version. If yes, what's the previous version?

msadlocha31 commented 4 weeks ago

We had previously 8.10 and hence tried to run the script that you recommended but it failed (see attached) image

Many Thanks

mohanbaabu1996 commented 4 weeks ago

Which version of Overwatch were you using at the time of running the upgrade script?

msadlocha31 commented 4 weeks ago

8.10 at the time of that run

mohanbaabu1996 commented 4 weeks ago

can you run the 0811 upgrade script using the cluster with the library 0811 itself?

msadlocha31 commented 4 weeks ago

I tried but the error for 8.11 lib is the same image

mohanbaabu1996 commented 3 weeks ago

Hi @msadlocha31

Please let me know if you're available for a 15-minute call. I'm from the IST timezone.

msadlocha31 commented 3 weeks ago

Hi @mohanbaabu1996 , sure, let me know when you have time.

I'm in the CET zone. I'm free tomorrow after 9:30 (so 13:00 your time, I think) for 1.5 hours, or Wednesday after 10:00 (13:30 your time).

If none of these times fit you, please propose something else.

mohanbaabu1996 commented 3 weeks ago

Thanks, I can meet tomorrow at 9:30 AM CET (13:00 PM IST) on Tuesday. Can you please send an invite to mohan.baabu@databricks.com for 30 minutes ? Thanks!

mohanbaabu1996 commented 3 weeks ago

Hi, @msadlocha31 ,

We closed this ticket and were able to fix the 0811 upgrade script. Thanks!