Added flush_sdout_after_log_entry kwag to dp logger. By default it is set to false. when it is set to true it will flush the buffer after each logging statement
Additional Info: "logs are getting concatenated (so 2 or more logging statements can get sent to cloudwatch as a single event via glue). This is messy and will cause a lot of issues if/when we move into the world of logging to elastic search/kibana.
This is happening because more than one log message gets sent to the buffer (memory for standard out) before it gets flushed (printed to console)."
What
Added
flush_sdout_after_log_entrykwag to dp logger. By default it is set to false. when it is set to true it will flush the buffer after each logging statementAdditional Info: "logs are getting concatenated (so 2 or more logging statements can get sent to cloudwatch as a single event via glue). This is messy and will cause a lot of issues if/when we move into the world of logging to elastic search/kibana. This is happening because more than one log message gets sent to the buffer (memory for standard out) before it gets flushed (printed to console)."
How to review
check the logic or any typo's
Who can review
Anyone