Codium-ai / pr-agent

๐Ÿš€CodiumAI PR-Agent: An AI-Powered ๐Ÿค– Tool for Automated Pull Request Analysis, Feedback, Suggestions and More! ๐Ÿ’ป๐Ÿ”
Apache License 2.0
5.75k stars 534 forks source link

Allow to enable slack notifications for newly reviewed pr result #999

Closed jiezhangaofl closed 3 months ago

jiezhangaofl commented 3 months ago

User description

We've seen the use case where developers want to notify the team or code maintainers about newly reviewed pull requests. I know all git providers sort of already provide the slack integration. But this feature from the pr agent will include the review summary, like reivew effort and files changed. It would be helpful for code reviewers to get a quick idea on how the pr look like.
Users can enable or disable this feature with the slack config in the configuration.toml file, default to disabled.


PR Type

Enhancement, Configuration changes, Dependencies


Description


Changes walkthrough ๐Ÿ“

Relevant files
Enhancement
misc_features.py
Add Slack notification functionality for newly reviewed PRs

pr_agent/algo/misc_features.py
  • Added a function send_notification to send Slack notifications for
    newly reviewed PRs.
  • Integrated Slack SDK for sending messages.
  • Constructed a Slack message with PR details including review effort
    and files changed.
  • +40/-0   
    pr_reviewer.py
    Integrate Slack notifications into PR review process         

    pr_agent/tools/pr_reviewer.py
  • Imported send_notification function.
  • Called send_notification if Slack notifications are enabled in
    settings.
  • +4/-0     
    Configuration changes
    .secrets_template.toml
    Add Slack token configuration template                                     

    pr_agent/settings/.secrets_template.toml - Added Slack token configuration template.
    +4/-0     
    configuration.toml
    Add Slack configuration options                                                   

    pr_agent/settings/configuration.toml
  • Added Slack configuration options for enabling notifications and
    specifying the Slack channel.
  • +5/-1     
    Dependencies
    requirements.txt
    Add Slack SDK dependency                                                                 

    requirements.txt - Added `slack_sdk` dependency.
    +1/-0     

    ๐Ÿ’ก PR-Agent usage: Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    codiumai-pr-agent-pro[bot] commented 3 months ago

    PR Reviewer Guide ๐Ÿ”

    โฑ๏ธ Estimated effort to review [1-5] 3
    ๐Ÿ… Score 75
    ๐Ÿงช Relevant tests No
    ๐Ÿ”’ Security concerns No
    ๐Ÿ”€ Multiple PR themes
    Sub-PR theme:
    Add Slack notification functionality

    Relevant files:
    • pr_agent/algo/misc_features.py
    Sub-PR theme:
    Update configuration settings for Slack integration

    Relevant files:
    • pr_agent/settings/.secrets_template.toml
    • pr_agent/settings/configuration.toml
    โšก Key issues to review Possible Bug:
    The code in send_notification function checks if the PR has been reviewed by looking for a comment starting with "## PR Review". This might not be reliable if the format of the comment changes or if there are multiple comments that could match.
    Performance Concern:
    The send_notification function retrieves all comments and reverses them to check the latest one. This could be inefficient for PRs with many comments.
    codiumai-pr-agent-pro[bot] commented 3 months ago

    PR Code Suggestions โœจ

    CategorySuggestion                                                                                                                                    Score
    Possible bug
    Handle the case where review_effort is an empty list to avoid an IndexError ___ **The review_effort variable is being assigned a list from re.findall, but it is later
    accessed as if it were a string. This can lead to an IndexError if review_effort is empty.
    It would be safer to handle this case explicitly.** [pr_agent/algo/misc_features.py [20-29]](https://github.com/Codium-ai/pr-agent/pull/999/files#diff-5a97d8b4d7a93fb8a44bcb70a235587ac739cda05fadd749f1ba2d42d76b8978R20-R29) ```diff -review_effort = re.findall(r'\d+', review_effort) if review_effort else "N/A" +review_effort = re.findall(r'\d+', review_effort) if review_effort else [] +review_effort = review_effort[0] if review_effort else "N/A" message = """ @here MR: <{MRLink}|{Title}>\nAuthor: @{Author}\nReview Effort [1-5]: {ReviewEffort}\nFiles changed: {FilesChanged} """.format( Title=mr_details.title, - ReviewEffort=review_effort[0] if review_effort else "N/A", + ReviewEffort=review_effort, Author=mr_details.author['username'], FilesChanged=mr_details.changes_count, MRLink=git_provider.get_pr_url() ) ``` - [ ] **Apply this suggestion**
    Suggestion importance[1-10]: 8 Why: The suggestion correctly identifies a potential IndexError due to accessing an index of an empty list. The proposed fix correctly handles this scenario, which is crucial for preventing runtime errors.
    8
    Best practice
    Add exception handling for sending Slack messages to manage potential errors ___ **The send_notification function does not handle potential exceptions that might occur when
    sending a message to Slack. Adding a try-except block can help manage these exceptions
    gracefully.** [pr_agent/algo/misc_features.py [30-40]](https://github.com/Codium-ai/pr-agent/pull/999/files#diff-5a97d8b4d7a93fb8a44bcb70a235587ac739cda05fadd749f1ba2d42d76b8978R30-R40) ```diff -client.chat_postMessage( - channel=slack_channel, - blocks=[{ - "type": "section", - "text": { - "type": "mrkdwn", - "text": message - } - }], - text=message -) +try: + client.chat_postMessage( + channel=slack_channel, + blocks=[{ + "type": "section", + "text": { + "type": "mrkdwn", + "text": message + } + }], + text=message + ) +except Exception as e: + print(f"Failed to send Slack notification: {e}") ``` - [ ] **Apply this suggestion**
    Suggestion importance[1-10]: 7 Why: Adding exception handling for external API calls is a best practice and helps in managing errors gracefully. This suggestion correctly identifies a potential improvement area in error handling.
    7
    Performance
    Fetch settings once and reuse them to improve efficiency ___ **The send_notification function currently fetches settings multiple times. It would be more
    efficient to fetch the settings once and reuse them.** [pr_agent/algo/misc_features.py [8-12]](https://github.com/Codium-ai/pr-agent/pull/999/files#diff-5a97d8b4d7a93fb8a44bcb70a235587ac739cda05fadd749f1ba2d42d76b8978R8-R12) ```diff -slack_channel = get_settings().slack.slack_channel +settings = get_settings().slack +slack_channel = settings.slack_channel if slack_channel: - slack_token = get_settings().slack.token - client = WebClient(token=slack_token) + client = WebClient(token=settings.token) ``` - [ ] **Apply this suggestion**
    Suggestion importance[1-10]: 6 Why: The suggestion improves efficiency by reducing redundant calls to `get_settings()`. This is a good practice for optimizing performance, although it's not critical.
    6
    Possible issue
    Ensure required keys are present in data before calling send_notification to avoid potential KeyError ___ **The send_notification function call should be placed after checking if data contains the
    required keys to avoid potential KeyError.** [pr_agent/tools/pr_reviewer.py [200-202]](https://github.com/Codium-ai/pr-agent/pull/999/files#diff-8e265068e189a06852605cc694b03e92b523b4f8162077a2f3455ced4cdad8dcR200-R202) ```diff -if get_settings().slack.enabled: - send_notification(data, self.git_provider) if 'code_feedback' in data: + if get_settings().slack.enabled: + send_notification(data, self.git_provider) ``` - [ ] **Apply this suggestion**
    Suggestion importance[1-10]: 6 Why: The suggestion aims to prevent a potential KeyError by ensuring necessary data is available before proceeding with the function call. This is a valid concern, but the actual implementation might need to consider the overall logic and dependencies in the code.
    6
    mrT23 commented 3 months ago

    Hi @jiezhangaofl , thanks for the PR

    In its current form, it is suitable for a local fork, and cannot be merged as a general PR-agent feature.

    It contains multiple deal-breaker problems: 1) it is gitlab-only, and will crash pr-agent in other providers 2) It does extra calls to the git provider API, something we want to avoid 3) It has a very specific design, which combines PR-agent feedback for different tools in a way I don't think its optimal for a typical user

    Hence, to consider this as a feature of pr-agent we can actually deploy, it will need the following changes: to any git provider, deploy the final feedback of the 'review' tool, as-is, to a designated slack channel

    If you will make these changes, we can consider merging and deploying it. If you prefer it in its current form, it is suitable for your local fork.

    jiezhangaofl commented 3 months ago

    @mrT23 Thanks for the feedback!
    It's my miss that I'm using gitlab specific code. Now that's removed and I actually feel it makes more sense to post every new result results. Could you take a peek again?

    mrT23 commented 3 months ago

    @jiezhangaofl read my entire feedback, and address all the parts. Despite the changes It is still Gitlab only (git_provider.mr), and it still "refactors" the response of the tool in a way that is not aligned with the pr-agent 'review' tool design.