Fine-tune CI Workflows in PR

[binary-ho]

1. What this PR does / why we need it:

The execution time of the jobs included in the CI was too long. We had to wait longer to receive results due to unnecessary tests.

• Build and Integration Test: 8 mins
• Benchmark: 5 mins
• Sharding Test: 3 mins

To prevent unnecessary CI tests from running, each CI test now only runs when there are changes related to the corresponding code.

1. Now, you can check the CI results faster because only the relevant tests for the changed code are executed.
2. You can easily verify whether a test was executed in the summary. (Refer to the image below)
   • Green check: Job executed
   • Red X: Job failed
   • Gray circle: Test did not run because it wasn’t the target package

Target packages:

• build: all (except those ignored by paths-ignore)
• benchmark: 'pkg/', 'server/packs/'
• sharding test: 'server/backend/database/**'

2. Which issue(s) this PR fixes

Fine-tune CI Workflows in PR fixes #954

3. Special notes for your reviewer

Please check the target packages for each CI job.

Although I tried to classify them on my own, there are a few parts that feel ambiguous.

3.1 Build test target

There’s no specific target in the CI yml file, but I added yorkie/design/** to paths-ignore.

If there was a particular reason for not including the design package in the related issue #873, please let me know!

As a result, changes to the following targets will be ignored. (No CI tests will run)

If there are any other targets that should be added, please let me know.

  pull_request:
    branches: [ main ]
    paths-ignore:
      - 'api/docs/**'
      - 'build/charts/**'
      - 'design/**'
      - '**/*.md'
      - '**/*.txt'
      - '**/.gitignore'

Additionally, a job has been added to check the packages affected by changes.

jobs:
  ci-target-check:
    runs-on: ubuntu-latest

    outputs:
      build: ${{ steps.ci-target-check.outputs.build }}
      bench: ${{ steps.ci-target-check.outputs.bench }}
      sharding-test: ${{ steps.ci-target-check.outputs.sharding-test }}

    steps:
    - name: CI target check by path
      uses: dorny/paths-filter@v3
      id: ci-target-check
      with:
        filters: |
          build: '**'
          bench: 
            - 'pkg/**'
            - 'server/packs/**'
          sharding-test:
            - 'server/backend/database/**'

And each job checks the flag values as follows and executes the job if the value is true.

build:
    needs: ci-target-check
    if: ${{ needs.ci-target-check.outputs.build == 'true' }}

bench:
    needs: ci-target-check
    if: ${{ needs.ci-target-check.outputs.bench == 'true' }}

sharding_test:
    needs: ci-target-check
    if: ${{ needs.ci-target-check.outputs.sharding-test == 'true' }}

3.2 Bench test target

By default, all packages under yorkie/pkg are included. I marked these with O in the diagram below.

The parts that felt ambiguous are marked with ? in the diagram, and I’ve left additional questions.

3.2.1 (Question) Packages that need confirmation

Should the following packages be included or excluded in the benchmark targets?

• yorkie/client package → Since it mainly serves as a layer for making calls like yorkie/api, I want to exclude it. (I have excluded it for now.)
• yorkie/server/packs package → This package defines functions for pushing or pulling changes. I believe only server/packs within the server package should be included in the bench target. Is this correct? (I have included it for now.)

3.2.2 Packages excluded from the bench test target

• yorkie/api → Excluded from the bench target since it only makes API calls
• yorkie/cmd, yorkie/internal, yorkie/design → Excluded from the bench target
• yorkie/server (excluding the yorkie/server/packs package)

3.3 Shard test target

• yorkie/server/backend/database

3.4 Test Result

I tested to ensure everything works correctly.

3.4.1 Build test

Only the build will be executed when packages unrelated to CRDT and shard-test are modified.

→ Github Action Result: https://github.com/yorkie-team/yorkie/actions/runs/10387699150

3.4.2 Bench test

After modifying the packs.go file, both the build test and benchmark were executed out of the three jobs.

(The build failure was due to adding a TODO comment for testing purposes)

→ Github Action Result: https://github.com/yorkie-team/yorkie/actions/runs/10387811024

3.4.3 Sharding-test

After modifying the server/backend/database/memory/database.go file, the sharding test was additionally executed.

(The build failure was due to adding a TODO comment for testing purposes)

→ Github Action Result: https://github.com/yorkie-team/yorkie/actions/runs/10387851119

Summary by CodeRabbit

• New Features

  • Introduced a new job in the CI workflow for target checks, optimizing the build process by conditionally executing jobs based on relevant file changes.

• Improvements

  • Streamlined job steps for clarity and efficiency, ensuring that only necessary builds and tests run, enhancing overall CI performance.

[coderabbitai[bot]]

Walkthrough

The CI workflow has been significantly enhanced to improve efficiency and clarity. A new job, ci-target-check, has been introduced to conditionally determine which parts of the codebase require builds, benchmarks, or tests based on recent changes. This optimization results in streamlined execution of the build, bench, and sharding-test jobs, ensuring they only run when relevant files are modified. Overall, these modifications foster a more responsive and maintainable CI process.

Changes

Files	Change Summary
.github/workflows/ci.yml	Added ci-target-check job to optimize CI workflow; reorganized steps; improved conditional execution for existing jobs (build, bench, sharding-test); clarified linting steps.

Poem

🐇 In the land of code, we hop and play,
With new checks in place, we brighten the day.
Builds run swift, tests dance with delight,
A streamlined CI, oh what a sight!
Bouncing through paths, we’re agile and free,
Celebrating changes, just you wait and see! 🌟

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

Tips

### Chat There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai): - Review comments: Directly reply to a review comment made by CodeRabbit. Example: - `I pushed a fix in commit .` - `Generate unit testing code for this file.` - `Open a follow-up GitHub issue for this discussion.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit testing code for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai generate interesting stats about this repository and render them as a table.` - `@coderabbitai show all the console.log statements in this repository.` - `@coderabbitai read src/utils.ts and generate unit testing code.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` - `@coderabbitai help me debug CodeRabbit configuration file.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (invoked as PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai full review` to do a full review from scratch and review all the files again. - `@coderabbitai summary` to regenerate the summary of the PR. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository. - `@coderabbitai help` to get help. Additionally, you can add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. ### CodeRabbit Configuration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information. - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json` ### Documentation and Community - Visit our [Documentation](https://coderabbit.ai/docs) for detailed information on how to use CodeRabbit. - Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback. - Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 51.06%. Comparing base (80c6ea0) to head (57ea87b). Report is 2 commits behind head on main.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #964 +/- ## ======================================= Coverage 51.06% 51.06% ======================================= Files 73 73 Lines 10782 10782 ======================================= Hits 5506 5506 Misses 4725 4725 Partials 551 551 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[krapie]

@binary-ho If you are still working on the PR and it is not ready for the review, I recommend to create PR in draft mode.

[binary-ho]

thank you for review kevin As discussed, I have organized the dependencies for the bench tests. As a result, the benchmark tests will be triggered by changes in the following packages. Could you please check this?

  bench: 
    - 'admin/**'
    - 'api/converter/**'
    - 'api/types/**'  
    - 'client/**'
    - 'pkg/document/**'
    - 'pkg/locker/**'
    - 'server/**'
    - 'test/helper/**'

1. Categorized by Dependency Package

1.1 Document

• "pkg/document"
  • "pkg/document/json"
  • "pkg/document/presence"
  • "pkg/document/change"
  • "pkg/document/key"
  • "pkg/document/time"
  • "pkg/document/crdt"
• "pkg/locker"

1.2 Server

• "server"
  • "server/backend"
  • "server/backend/database"
  • "server/backend/sync"
  • "server/documents"
  • "server/logging"
  • "server/packs"
  • "server/profiling"
  • "server/profiling/prometheus"
  • "server/clients"
  • "server/projects"
  • "server/rpc"

1.3 API

• "api/types"
• "api/converter"

1.4 Client

• "client"

1.5 Test

• "test/helper"

2. Dependencies Organized by Test

(For reference)

• document_bench_test

  • "pkg/document"

• grpc_bench_test

  • "admin"
  • "api/types"
  • "client"
  • "pkg/document"
  • "pkg/document/json"
  • "pkg/document/presence"
  • "server"
  • "server/backend/database"
  • "server/logging"
  • "test/helper"

• locker_bench_test

  • "pkg/locker"

• push_pull_bench_test

  • "api/converter"
  • "api/types"
  • "pkg/document"
  • "pkg/document/change"
  • "pkg/document/json"
  • "pkg/document/key"
  • "pkg/document/presence"
  • "pkg/document/time"
  • "server/backend"
  • "server/backend/database"
  • "server/documents"
  • "server/packs"
  • "server/profiling/prometheus"
  • "test/helper"

• sync_bench_test

  • "server/backend/sync"

• text_editing_bench_test

  • "api/types"
  • "client"
  • "server/backend"
  • "server/backend/database"
  • "server/clients"
  • "server/profiling"
  • "server/profiling/prometheus"
  • "server/projects"
  • "server/rpc"

• tree_editing_bench_test

  • "api/converter"
  • "pkg/document/crdt"
  • "pkg/document/json"
  • "test/helper"

[hackerwins]

@krapie , @binary-ho

For some reason the CI was broken in main. Now I've rolled back the main branch to the previous commit.

https://github.com/yorkie-team/yorkie/actions/runs/10402716854

[binary-ho]

The Reason build failed is that the checkout step was missing in the ci-target-check job.

The following step should be added:

   - name: Check out code
    uses: actions/checkout@v4

The ci-target-check job should be updated as follows:

jobs:
  ci-target-check:

    ...
    steps:
    - name: Check out code
    uses: actions/checkout@v4

    - name: CI target check by path
      uses: dorny/paths-filter@v3
      id: ci-target-check
      with:
        filters: |
          build: '**'
          bench: 
            - 'pkg/**'
            - 'server/**'
            - 'client/**'
            - 'admin/**'
            - 'api/converter/**'

          sharding-test:
            - 'server/backend/database/**'

The reason why the PR build succeeded was that “dorny/paths-filter@v3” did not require a checkout in the PR.

I misunderstood this comment and removed it entirely.

This issue was found by @devleejb. and @krapie has confirmed the issue and will take action to address it.

Thank you, devleejb and krapie

[krapie]

@binary-ho I will merge the following PR: https://github.com/yorkie-team/yorkie/pull/965 soon. Therefore, you don't have to take further actions :)