aws-samples / dbt-glue

This repository contains the dbt-glue adapter
Apache License 2.0
96 stars 67 forks source link

Pagination bug #342

Closed danphenderson closed 7 months ago

danphenderson commented 7 months ago

resolves #58

Description

Using the get_paginator to make a continuation call over all databases defined in the targeted glue catalog.

Checklist

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

aajisaka commented 7 months ago

@danphenderson Would you update the CHANGELOG.md file to include your bug fix?

danphenderson commented 7 months ago

@aajisaka Done! I also caught one more instance of the bug and updated the PR.

On a side note, I had to bypass the pre-commit hooks with the --no-verify git commit flag as there are a lot of pre-commit issues that should be resolved in an isolated PR. I will file a separate issue for that. Also, currently, we are swallowing errors after logging them in imply.py, which is likely not the best course of action (do what you will with that observation).

aajisaka commented 7 months ago

Thank you @danphenderson for your update.

Also, please feel free to file separate issue/PR for your findings.