M2 - A4 - Create additional features, such as time-based features or statistical aggregates, to capture temporal patterns and trends in the data

[renovus]

Ticket Description:

Objective: Implement additional features, including time-based features and statistical aggregates, to capture temporal patterns and trends in the dataset. The goal is to enhance the dataset's richness by introducing features that encapsulate temporal dependencies and provide valuable context for predictive modeling.

Tasks:

1. Temporal Features:

   • Introduce time-based features such as hour, month, day, and season to capture daily, monthly, and seasonal patterns in the data.
   • Utilize datetime functionalities to extract relevant time components from timestamps.

2. Statistical Aggregates:

   • Generate statistical aggregates such as mean, standard deviation, maximum, minimum, and exponentially weighted moving average (EMA) for target and environmental variables over different time windows.
   • Apply rolling window-based computations to capture temporal trends and variations within the dataset.

3. Feature Creation:

   • Implement feature creation techniques to derive new variables that encapsulate temporal context and dependencies.
   • Ensure alignment with the dataset's domain knowledge and relevance to predictive modeling objectives.

4. Integration with Existing Features:

   • Integrate newly created features with existing variables to enrich the dataset and provide a comprehensive representation of temporal dynamics.
   • Ensure compatibility and coherence between newly introduced features and the dataset's structure.

5. Documentation and Testing:

   • Document the implementation process, including code logic and methodologies employed for feature creation.
   • Test the functionality of new features to ensure accuracy and consistency in capturing temporal patterns and trends.

Deliverables:

• Updated Jupyter Notebook or Python script showcasing the implementation of additional features.
• Description of newly introduced features, including their purpose and methodology for creation.
• Documentation detailing the integration of new features with existing variables and their relevance to predictive modeling objectives.
• Testing results demonstrating the accuracy and effectiveness of new features in capturing temporal patterns and trends.

[fcggamou]

Related commits: https://github.com/Renovus-Tech/solarec-python/commit/255706dab4e0168fcd44f026ee0644b8d910d60e https://github.com/Renovus-Tech/solarec-python/commit/47b5241a254b40cee0a2b63077f95dcb86a4f468 https://github.com/Renovus-Tech/solarec-python/commit/25897914f663cfcacae7f5161ce4ef4c70fc1d41

[fcggamou]

Related notebook: https://github.com/Renovus-Tech/solarec-python/blob/main/app/ml/notebooks/anomaly_detection/2%20-%20Feature%20Engineering.ipynb