dbt-labs / dbt-athena

The athena adapter plugin for dbt (https://getdbt.com)

https://dbt-athena.github.io

Apache License 2.0

228 stars 100 forks source link

feat: Implement iceberg retry logic #657

Closed svdimchenko closed 5 months ago

svdimchenko commented 6 months ago

Description

Implemented num_iceberg_retries parameters

Checklist

[x] You followed contributing section
[x] You kept your Pull Request small and focused on a single feature or bug fix.
[ ] You added unit testing when necessary
[ ] You added functional testing when necessary

nicor88 commented 6 months ago

Code looks good, I'm wondering if we can use the test suite for functional testing to attempt to run few incremental models with append behavior to simulate a concurrent insert scenario.

nicor88 commented 5 months ago

@svdimchenko if you have some urgency on merging this, I'm fine to to merge as it is, because you did the functional testing in your environment. It will be a nice to have to add the functional testing of running the same "model" concurrently, but it's not really necessary before the merging.

svdimchenko commented 5 months ago

@svdimchenko if you have some urgency on merging this, I'm fine to to merge as it is, because you did the functional testing in your environment. It will be a nice to have to add the functional testing of running the same "model" concurrently, but it's not really necessary before the merging.

I've managed to add functional tests, but somehow functional tests are not run in ci as they are marked as skipped(

nicor88 commented 5 months ago

@svdimchenko I noticed in other prs too that the functional tests were skipped, they should kick in now. I don't know what lead to that issue, but I removed the label and added it again