search

moj-analytical-services / etl_manager

A python package to create a database on the platform using our moj data warehousing framework
21 stars 8 forks source link

Add glue 4.0 as a valid glue version #151

Closed Staberinde closed 7 months ago

Staberinde commented 7 months ago

According to https://docs.aws.amazon.com/glue/latest/dg/migrating-version-40.html#migrating-version-40-from-30 the API Interface remains unchanged between 3.0 and 4.0

Staberinde commented 7 months ago

For some reason all your CI pipelines seem to be running against python 3.12, which removes deprecated APIs needed by setuptools==59.6.0, which is the newest version which will support python 3.6 (which is specified as being the lower bound in pyproject.toml:

...
  File "/opt/hostedtoolcache/Python/3.12.0/x64/lib/python3.12/site-packages/pkg_resources/__init__.py", line 2172, in <module>
...

~I'm not sure fixing that is in the scope of this PR. Let me know if you want me to revert the poetry.lock changes.~

Turns out it was an easy fix.

Staberinde commented 7 months ago

@jhpyke should I increment the version in pyproject.toml/add changelog as part of this PR or separately? If part of this PR, would you agree that this is a minor version increment?

jhpyke commented 7 months ago

My instinct for cleanliness would be to have a single explicit 'release' PR where you update the pyproject, incorporating the updates to the Release Notes as well, but I think it's perfectly reasonable if you want to do it as part of this PR. I would agree with you that this represents a 'minor' level change - textbook add[ing] functionality in a backward compatible manner work based on the AWS release notes.