Getting-started-with-AWS-Glue-and-Python-Pyspark-for-Beginners

Tutorials

• Getting started with AWS Glue and Python (Pyspark) for Beginners #1

• https://www.youtube.com/watch?v=XeqWKn1jOI8&t=134s

• Getting started with AWS Glue and Python (Pyspark) for Beginners #2 Pyspark Basic

• https://www.youtube.com/watch?v=lVdvOltkg8c

• Getting started with AWS Glue and Python (Pyspark) for Beginners #3

• https://www.youtube.com/watch?v=_VgWkKIhU34

• Getting started with AWS Glue and Python (Pyspark) for Beginners #4 Crawler Infrasture code

• https://www.youtube.com/watch?v=GVjp8pTAsXE&t=55s

• Getting started with AWS Glue and Python (Pyspark) for Beginners #5 Glue Script

• https://www.youtube.com/watch?v=u6GDqPzY5Hg&feature=youtu.be

• Getting started with AWS Glue and Python (Pyspark) for Beginners #6 Glue Script Infrasture code

• https://www.youtube.com/watch?v=hhuqFwhP-_Y

• Getting started with AWS Glue and Python (Pyspark) for Beginners #7 Schedule Glue Script

• https://www.youtube.com/watch?v=4LJ-rLI_tA4&t=78s

• Getting started with AWS Glue and Python (Pyspark) for Beginners #8 Source Schema Change

• https://www.youtube.com/watch?v=XdUhGmAOn9E

• Resource : https://aws.amazon.com/blogs/big-data/identify-source-schema-changes-using-aws-glue/

Commands

npx serverless config credentials --provider aws --key XXX  --secret XXX -o
npx sls deploy

Watch More

Step by Step guide How to Move data from DynamoDB to Aurora Postgres SQL with AWS Glue ETL https://www.youtube.com/watch?v=zWoXB6-w58A&list=PLL2hlSFBmWwyHxy3rH2lltZxPeOUMYQEq

Step by Step Guide How to Move Data with CDC from Datalake S3 to AWS Aurora Postgres Using Glue https://www.youtube.com/watch?v=orze49fvTZo&list=PLL2hlSFBmWwyHxy3rH2lltZxPeOUMYQEq&index=2

Step by Step Guide How to Move Data with CDC from Datalake S3 to AWS Aurora Postgres Using Glue https://www.youtube.com/watch?v=orze49fvTZo&list=PLL2hlSFBmWwyHxy3rH2lltZxPeOUMYQEq&index=3

Identify source schema changes using AWS Glue On Datalake AWS S3 | Demo https://www.youtube.com/watch?v=dfLb_kTNbEM&list=PLL2hlSFBmWwyHxy3rH2lltZxPeOUMYQEq&index=4

How to Flatten Complex JSON into Separate Folders for Athena Query |Glue Job | Glue Notebook https://www.youtube.com/watch?v=47zL8cP6ZUI&list=PLL2hlSFBmWwyHxy3rH2lltZxPeOUMYQEq&index=5

How to Move Data from DynamoDB to Data lake S3 | Hands on Lab| Glue |Serverless Framework| Lab 23 https://www.youtube.com/watch?v=V3O9O5plC4M&list=PLL2hlSFBmWwyHxy3rH2lltZxPeOUMYQEq&index=6

Using AWS Glue or comprehend, redact PII identified text from Data Lake (AWSS3) (HIPA) Demo | https://www.youtube.com/watch?v=b5mA6GsHSmw&list=PLL2hlSFBmWwyHxy3rH2lltZxPeOUMYQEq&index=7

When your Glue ETL processes fail, how to get email alerts Detailed instructions with a code https://www.youtube.com/watch?v=O9k0w52AKUQ&list=PLL2hlSFBmWwyHxy3rH2lltZxPeOUMYQEq&index=8

Data Warehouse Glue Ingestion Architecture| SQL Server | DynamoDB| MongoDB | S3 | https://www.youtube.com/watch?v=fcVgl0T6TYQ&list=PLL2hlSFBmWwyHxy3rH2lltZxPeOUMYQEq&index=11