Closed amindadgar closed 1 month ago
The recent changes primarily focus on improving the logging and error reporting capabilities of the ETL processes for Discord data. The platform_id
parameter was removed from the transform
method. Logging statements were added at various stages to enhance observability and debugging, and error logging was updated to include message IDs.
File Path | Change Summary |
---|---|
dags/analyzer_helper/discord/discord_transform_raw_data.py |
Removed the platform_id from the transform method and updated error logging to include the message ID. |
dags/discord_analyzer_etl.py |
Added logging statements before and during the extraction, transformation, and loading stages of the ETL process for both raw data and raw members, and removed platform_id . |
In code, a change does swiftly bloom,
Logs now light the ETL room. 🌟
Withplatform_id
now set free,
Error messages speak with glee.
Transforming data, raw and bright,
Message IDs now in sight.
ETL flows with newfound might. 🚀
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
Summary by CodeRabbit
New Features
Chores