updates for python3 support in glue

awslabs / athena-glue-service-logs

Glue scripts for converting AWS Service Logs for use in Athena

Apache License 2.0

142 stars 46 forks source link

updates for python3 support in glue #16

Closed ryandeivert closed 3 years ago

ryandeivert commented 5 years ago

Description of changes:

AWS Glue recently launched support for python3

Including updates to convert this project to python3. The changes include the following:

update requirements for python3
converting all package code to python3
updating tests for python3
bumping version (unsure what this should actually be) and updating makefile to get version correctly

Testing:

I ensured all tests are still passing using make test

[x] By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

dacort commented 5 years ago

@ryandeivert This is awesome, thank you! Been on my backlog for a bit, so I definitely appreciate the contribution.

Will take a look at this within the next week.

juanqui commented 4 years ago

@ryandeivert This is awesome, thank you! Been on my backlog for a bit, so I definitely appreciate the contribution.

Will take a look at this within the next week.

Any updates on this?

dacort commented 4 years ago

Hi @juanqui - I unfortunately don't work at AWS anymore so not able to merge this in.

I'll try to ping somebody on my old team there, but no guarantees. Feel free to fork this, though!

sheyd commented 4 years ago

I did a manual zip deploy of this and ran against S3 access logs using Glue 2.0 Python3 with G.1X nodes, and it worked successfully, both for initial run, and diff updates against latest partitions. @ryandeivert Thanks for putting this PR in and @dacort for putting this code together to begin with!

serverlesspolska commented 3 years ago

Any news on this PR? Why it has not been merged in?

dacort commented 3 years ago

@serverlesspolska I was the primary maintainer of this repo, but then left Amazon and wasn't able to find a replacement. That said, I'm back at Amazon now so will take a look at what needs to be done to update this / if we should continue maintaining it. A lot has changed in the past year. :)

dacort commented 3 years ago

Thanks @ryandeivert for submitting this PR and I apologize for taking checks notes almost 2 years to merge it in.

Keep an eye on the repo as there have been a lot of changes in Glue since then and Blueprints/Workflows may be the path forward.