databrickslabs / overwatch

Capture deep metrics on one or all assets within a Databricks workspace
Other
225 stars 63 forks source link

Missing tables on overwatch #1035

Closed hazearias closed 9 months ago

hazearias commented 12 months ago

Sensitive Content Alert Please remember that this repository is publicly visible so do not post anything sensitive in your report such as user emails, secrets, etc.

Overwatch Version (com.databricks.labs:overwatch_2.12:0.7.2.0.2)

Describe the bug Missing tables related with jobs

Screenshots image

Issue Reports Please provide the reports requested from the Troubleshooting page in the docs.

Arfa-1999 commented 12 months ago

Hi,

Could you kindly share the pipeline report? To do so, run the command 'select * from ETL_DB.pipeline_report order by pipeline_snapTs desc;' to generate the report. Thank you!

Regards, Syeda Arfa

mateuszkulak commented 12 months ago

Hello, I'm working on this project with @hazearias. Unfortunately, we cannot send you this report here due to our company's sensitive data policy. Is there any alternative for that?

williamdphillips commented 11 months ago

@hazearias I faced the same issue with DBR 12.2 LTS but resolved it by switching the 11.3 LTS. Can you confirm which DBR you are using?

We also had to run the job a few times for the tables to all come up. After the first run, only a few of the tables in the consumer db tables were created/populated.

hazearias commented 11 months ago

@williamdphillips In Fact is the same issue, I tried to change the cluster but still have the same error for I few time.

I know that is working right now, Can be due the change of the cluster or work from Guenia?

With the purpose of documenting this issue and not replicate the error in the automation of the implementation @gueniai do you know what was the root cause and how to avoid it?

mohanbaabu1996 commented 9 months ago

Hi @hazearias ,

I hope you're doing good and everything is working fine. Please let us know we can close the ticket, if you don't have any issues Thank you!

mohanbaabu1996 commented 9 months ago

Hi @hazearias,

Kindly let us know if you've got any issues. Thank you!