Added Timestamp to dplogger and flushing after every log.
Hypothesis for the Glue logs not separating issue. AWS Glue Job is generating the logs at the same time and the logs are not being flushed
The structlog.processors.TimeStamper processor adds distinct timestamps to each log entry which will help with separating logs additionally added FlushStreamHandler class which will helps ensure that logs won't be batched and combined.
What
Added Timestamp to dplogger and flushing after every log.
Hypothesis for the Glue logs not separating issue. AWS Glue Job is generating the logs at the same time and the logs are not being flushed
The structlog.processors.TimeStamper processor adds distinct timestamps to each log entry which will help with separating logs additionally added FlushStreamHandler class which will helps ensure that logs won't be batched and combined.
How to review
Check the logic or is it the right approach.
Who can review
Anyone