For Unity Catalog Export functionality in Azure Databricks, create external table in Unity Catalog fails first time with "table already exists" error message but it succeeds in 2nd attempt.
It seems to basically the first table, the first time it tries the command, it says it failed because the table already exists. But then when we look at the command text, it runs it a second time and then it succeeds the second time around. The other thing is obviously, if we have a bunch of tables being updated at the same time and it generates a whole bunch of actions doing the same thing and some of them end up colliding and overlapping and they all fail obviously.
Bottom line - it’s functional, but its just generating a lot of errors and exceptions and failures, things like that get logged kind of mostly on the lakeFS side, i guess in the action logs, but also on the sql warehouse, we can see obviously all those failed commands.
What happened?
For Unity Catalog Export functionality in Azure Databricks, create external table in Unity Catalog fails first time with "table already exists" error message but it succeeds in 2nd attempt.
It seems to basically the first table, the first time it tries the command, it says it failed because the table already exists. But then when we look at the command text, it runs it a second time and then it succeeds the second time around. The other thing is obviously, if we have a bunch of tables being updated at the same time and it generates a whole bunch of actions doing the same thing and some of them end up colliding and overlapping and they all fail obviously.
Bottom line - it’s functional, but its just generating a lot of errors and exceptions and failures, things like that get logged kind of mostly on the lakeFS side, i guess in the action logs, but also on the sql warehouse, we can see obviously all those failed commands.