search

airqo-platform / AirQo-api

This repo contains API definitions for the AirQo platform
MIT License
21 stars 17 forks source link

Setup device weather data downloads #3226

Closed Mnoble-19 closed 3 weeks ago

Mnoble-19 commented 4 weeks ago

WHAT DOES THIS PR DO?

HOW DO I TEST OUT THIS PR?

WHICH ENDPOINTS SHOULD BE READY FOR TESTING?:

Summary by CodeRabbit

codecov[bot] commented 4 weeks ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 30.30%. Comparing base (ec293ad) to head (253328f).

Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/airqo-platform/AirQo-api/pull/3226/graphs/tree.svg?width=650&height=150&src=pr&token=HHq3qS3cL6&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=airqo-platform)](https://app.codecov.io/gh/airqo-platform/AirQo-api/pull/3226?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=airqo-platform) ```diff @@ Coverage Diff @@ ## staging #3226 +/- ## ======================================== Coverage 30.30% 30.30% ======================================== Files 184 184 Lines 24487 24487 Branches 3205 3205 ======================================== Hits 7421 7421 Misses 16952 16952 Partials 114 114 ```
github-actions[bot] commented 4 weeks ago

Analytics changes in this PR available for preview here

BenjaminSsempala commented 4 weeks ago

@coderabbitai review

coderabbitai[bot] commented 4 weeks ago
Actions performed Review triggered. > Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.
coderabbitai[bot] commented 4 weeks ago

Walkthrough

The recent updates across various modules in the analytics API focus on enhancing data processing and organization, improving error handling, and reformatting code for consistency and readability. Enhancements include better structuring of JSON responses, handling NaN values, adding weather-related fields, and updating logging configurations. Import statements have been reformatted, and some syntax adjustments have been made for improved clarity and maintainability.

Changes

File(s) Change Summary
src/analytics/api/models/base/data_processing.py Reformat imports, adjust error handling, enhance air quality data function, and modify JSON response.
src/analytics/api/models/base/diurnal_data_processing.py Reformat imports, update logging configuration, adjust error messages, and enhance response data.
src/analytics/api/models/events.py Add weather fields import and handling in download function.
src/analytics/api/utils/data_formatters.py Add comma to the end of params dictionary in a requests.post call.
src/analytics/api/utils/pollutants/pm_25.py Introduce WEATHER_FIELDS_MAPPER dictionary.
src/analytics/api/utils/pollutants/report.py Standardize quotes for strings and dictionary keys, reformat groupings and assignments.
src/analytics/api/views/data.py Add and handle new parameter weatherFields in request payload.
src/analytics/config.py Add spaces around assignment operator and introduce additional blank lines.
src/analytics/manage.py Update method declaration to use double quotes for methods parameter.

Sequence Diagrams

sequenceDiagram
    participant Client
    participant API Gateway
    participant DataExportResource
    participant BigQueryService
    participant DataProcessor

    Client->>API Gateway: POST /data/export request (with weatherFields)
    API Gateway->>DataExportResource: Forward request
    DataExportResource->>BigQueryService: download_from_bigquery(weatherFields)
    BigQueryService->>DataProcessor: Process data
    DataProcessor->>BigQueryService: Return processed data
    BigQueryService->>DataExportResource: Return data
    DataExportResource->>Client: Return processed response

Poem

In the land of code, where the data flows,
We polished and enhanced, as the analytics grows.
Weather fields now blend, in our query they are found,
JSONs neatly structured, errors handled all around.
NaNs won't hide, and logs now clear,
Our data dance refined, in this update we cheer! 🌟

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share - [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)
Tips ### Chat There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai): - Review comments: Directly reply to a review comment made by CodeRabbit. Example: - `I pushed a fix in commit .` - `Generate unit testing code for this file.` - `Open a follow-up GitHub issue for this discussion.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit testing code for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai generate interesting stats about this repository and render them as a table.` - `@coderabbitai show all the console.log statements in this repository.` - `@coderabbitai read src/utils.ts and generate unit testing code.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` - `@coderabbitai help me debug CodeRabbit configuration file.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (invoked as PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai full review` to do a full review from scratch and review all the files again. - `@coderabbitai summary` to regenerate the summary of the PR. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository. - `@coderabbitai help` to get help. Additionally, you can add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. ### CodeRabbit Configration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information. - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json` ### Documentation and Community - Visit our [Documentation](https://coderabbit.ai/docs) for detailed information on how to use CodeRabbit. - Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback. - Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.
coderabbitai[bot] commented 3 weeks ago
Actions performed Review triggered. > Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.
Mnoble-19 commented 3 weeks ago

@coderabbitai review

My guy, review the code yourself. Don't delegate to AI 😂

github-actions[bot] commented 3 weeks ago

Analytics changes in this PR available for preview here

coderabbitai[bot] commented 3 weeks ago

[!TIP] For best results, initiate chat on the files or code changes.

@BenjaminSsempala, I've identified the files that have been modified in this PR. Here's a brief overview of the key changes and some general recommendations:

Modified Files:

  1. Data Processing Modules:

    • data_processing.py and diurnal_data_processing.py: Ensure that new data processing functions are optimized for performance and include error handling. It's also important to maintain consistency with the existing codebase in terms of coding style and data structures used.
    • events.py: If new events or event handling logic has been added, verify that they integrate smoothly with the existing event management system.

  2. Utility Functions:

    • data_formatters.py, pm_25.py, report.py: Check for the correct implementation of any new data formatting logic, particularly how data is prepared for export. Make sure unit tests cover edge cases and data anomalies.

  3. API Views:

    • data.py: This file likely contains the endpoint logic for data downloads. Review for security concerns like proper authentication and data validation. Ensure that the endpoint is robust against common web vulnerabilities.

  4. Configuration and Management:

    • config.py: Any new configurations added should be documented and default values should be secure and sensible.
    • manage.py: If modified, ensure that new management commands are well-documented and error-free.

Recommendations:

Please let me know if you need a more detailed code analysis or if there are specific areas you want me to focus on!

Analysis chain ---
Scripts executed The following scripts were executed for the analysis: --- Script: ```shell #!/bin/bash # List files changed in the PR related to the analytics data download functionality git diff --name-only HEAD $(git merge-base HEAD main) ``` Length of output: 91 --- --- Script: ```shell #!/bin/bash # List files changed in the PR related to the analytics data download functionality, using the correct base branch git diff --name-only HEAD $(git merge-base HEAD staging) ``` Length of output: 452
coderabbitai[bot] commented 3 weeks ago

[!TIP] For best results, initiate chat on the files or code changes.

@Mnoble-19, it seems there was an issue accessing the repository to review the changes directly. Could you please confirm that the repository settings allow access for this operation, or if there's another way you'd prefer I access the necessary files for review? 😊

Looking forward to diving into the code with you! 🐰

Analysis chain
Scripts executed The following scripts were executed for the analysis: --- Script: ```shell #!/bin/bash # Find and display the contents of backend files potentially related to data export or downloading functionalities within the analytics directory. fd "download|export" src/analytics --exec cat {} ``` Unrecoverable error: Failed to clone repository
Mnoble-19 commented 3 weeks ago

@Codebmk the code is setup to work the same way as pollutant selection works. From the front end, kindly set it up so that one specifies which weather fields they want (temperature, humidity, wind_speed) and then these are what are put in the body.

just as it is done for the pollutants.

you're welcome to reach out if anything isn't clear

Baalmart commented 3 weeks ago

thanks @Mnoble-19