This pull request fixes two bugs:
1) The split statement for extracting the packet id from the object key was flawed as there is an underscore in "sales_order". For that reason, I need to reference the second last element of the split string instead of the second.
2) There was an issue with writing transformed data to S3 in Parquet format due to type inference errors. The Lambda function encountered a ValueError when converting date and time columns to Parquet, due to inconsistent data types.
I have also added an additional case statement to handle unknown table names.
This pull request fixes two bugs: 1) The split statement for extracting the packet id from the object key was flawed as there is an underscore in "sales_order". For that reason, I need to reference the second last element of the split string instead of the second. 2) There was an issue with writing transformed data to S3 in Parquet format due to type inference errors. The Lambda function encountered a ValueError when converting date and time columns to Parquet, due to inconsistent data types.
I have also added an additional case statement to handle unknown table names.