feat: skip_workgroup_check setting to reduce AWS throttling

[amacal]

Description

The adapter performs a GetWorkGroup operation on each thread, which is later cached. When dbt build is started for 10 independent models, it issues 10 AWS requests to get the same information. If dbt is orchestrated via Airflow to run 32 dbt tasks concurrently, it sends 320 GetWorkGroup requests, causing throttling on the AWS side.

This PR introduces a skip_workgroup_check setting, which instructs dbt to skip checking if a WorkGroup contains an enforced output location.

Checklist

• [x] You followed contributing section
• [x] You kept your Pull Request small and focused on a single feature or bug fix.
• [x] You added unit testing when necessary
• [x] You added functional testing when necessary

[nicor88]

@amacal implementation looks good.

Please have a look at the ci, precommit checks are failing. Provide unit testing for is_work_group_output_location_enforced function cover the introduced feature.

Nice to have, and not mandatory, add some "integration tests" to fully test the feature in a real environment.

[nicor88]

@amacal plese remember what I suggested before:

• add unit tests
• add integration tests
• document the new parameter in the docs

Without all these 3 conditions, we cannot merge. Thanks

[amacal]

I added a unit test and documented new parameter in the documentation. I cannot do functional test, because I cannot push it via org AWS account.

[nicor88]

@svdimchenko @Jrmyy could you have a look when you have some time? I'm currently away from keyboard. Thanks

[nicor88]

@amacal implementation looks fine, have a look at the CI, there is a failure

[amacal]

Is there anything I need to do to have it merged?

[nicor88]

@amacal you changes looks good. As you can see repository ownership is being moved to dbtlabs, they are the official owners. @colin-rogers-dbt could you have a look here?