external_location makes run fail because of HIVE_PATH_ALREADY_EXISTS

[bmasciarelli]

Hi!

I'm trying to use the "externallocation" propery but I get this error when second model runs, as it's trying to use same path again: **Target directory for table 'table'_ already exists**.

I've setted the propery on the dbt_project.yml at a project level. If I set it at a model level, it works the first time but on the second run, i get the same error.

Am i doing something wrong?

For what i've seen, when using the default path, final table path is generated using a UUID but when overriding it, this UUID is not used.

Thanks in advance!

[Tomme]

Hey @bmasciarelli,

Apologies for the delay in my response. You are encountering this issue since you are using a static value for external_location. I would advise using a dynamic value if possible (similar to how default path is working with UUID).

Example dynamic values:

• Within .sql config block
  • external_location="s3://dbt-tables/example-directory/" + run_started_at.isoformat()
  • external_location="s3://dbt-tables/example-directory/" + invocation_id
• Within dbt_project.yml
  • +external_location: "s3://dbt-tables/example-directory/{{ invocation_id }}"

I have updated the README.md to reflect this recommendation so thank you for bring this to my attention 👍

Lastly, I do not automatically append a UUID to a custom external_location value, since I want to give flexiblity to the users of the adapter.

Tomme

[bmasciarelli]

No worries Tomme, thanks for the clarification and the amazing work you've done.