feat(api): Fetch total count of environments, secrets and variables in project

[unamdev0]

User description

Description

Sending total Count of env,variable and secrets when fetching project data for a workspace

Fixes #308

Screenshots of response

Further improvement

• [] Need to update postman Sample response

  "items": [
      {
          "id": "c6d92085-0e85-4986-a1d3-7923583efa22",
          "name": "Project 1",
          "slug": "project-1-oa2j9",
          "description": "Dummy project 1",
          "createdAt": "2024-09-13T15:45:42.014Z",
          "updatedAt": "2024-09-13T15:45:42.014Z",
          "storePrivateKey": true,
          "isDisabled": false,
          "accessLevel": "PRIVATE",
          "isForked": false,
          "lastUpdatedById": "cm0v65bm00000130sigtvry7y",
          "workspaceId": "3b41add3-4e14-4866-a6e8-0669d1d7b08b",
          "forkedFromId": null,
          "totalEnvironmentsOfProject": 3,
          "totalVariablesOfProject": 1,
          "totalSecretsOfProject": 1
      },
      {
          "id": "45dae533-8556-4621-b841-43502fe85752",
          "name": "Project 2",
          "slug": "project-2-o9rl4",
          "description": "Dummy project 1",
          "createdAt": "2024-09-13T16:25:40.877Z",
          "updatedAt": "2024-09-13T16:25:40.877Z",
          "storePrivateKey": true,
          "isDisabled": false,
          "accessLevel": "PRIVATE",
          "isForked": false,
          "lastUpdatedById": "cm0v65bm00000130sigtvry7y",
          "workspaceId": "3b41add3-4e14-4866-a6e8-0669d1d7b08b",
          "forkedFromId": null,
          "totalEnvironmentsOfProject": 3,
          "totalVariablesOfProject": 0,
          "totalSecretsOfProject": 0
      },
      {
          "id": "5392bcb8-dcd7-41d3-9c78-1adfa0818f1b",
          "name": "Project 3",
          "slug": "project-3-b8i47",
          "description": "Dummy project 1",
          "createdAt": "2024-09-13T16:27:25.038Z",
          "updatedAt": "2024-09-13T16:27:25.038Z",
          "storePrivateKey": true,
          "isDisabled": false,
          "accessLevel": "PRIVATE",
          "isForked": false,
          "lastUpdatedById": "cm0v65bm00000130sigtvry7y",
          "workspaceId": "3b41add3-4e14-4866-a6e8-0669d1d7b08b",
          "forkedFromId": null,
          "totalEnvironmentsOfProject": 1,
          "totalVariablesOfProject": 0,
          "totalSecretsOfProject": 0
      }
  ],
  "metadata": {
      "page": 0,
      "perPage": 10,
      "pageCount": 1,
      "totalCount": 3,
      "links": {
          "self": "/project/all/my-first-workspace-rbqs2?page=0&limit=10&sort=name&order=asc&search=project 1",
          "first": "/project/all/my-first-workspace-rbqs2?page=0&limit=10&sort=name&order=asc&search=project 1",
          "previous": null,
          "next": null,
          "last": "/project/all/my-first-workspace-rbqs2?page=0&limit=10&sort=name&order=asc&search=project 1"
      }
  }
  }

---

PR Type

Tests, Enhancement

---

Description

• Enhanced the project fetching functionality to include total counts of environments, variables, and secrets for each project.
• Implemented permission checks to ensure only authorized data is counted.
• Added end-to-end tests to verify the correct fetching of environments, variables, and secrets for projects within a workspace.

---

Changes walkthrough 📝

	Relevant files
Tests	project.e2e.spec.ts Add tests for fetching environments, variables, and secrets in projects apps/api/src/project/project.e2e.spec.ts Added a new project for testing environment, secret, and variable fetching. Implemented a test to verify fetching of all environments, variables, and secrets for projects in a workspace. Added setup for environments, secrets, and variables in the test. +91/-1
Enhancement	project.service.ts Enhance project fetching with total counts of environments, variables, and secrets apps/api/src/project/service/project.service.ts Modified project fetching to include total counts of environments, variables, and secrets. Implemented logic to calculate these totals with permission checks. Refactored project fetching logic to accommodate new data. +54/-1

---

💡 PR-Agent usage: Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[codiumai-pr-agent-free[bot]]

PR Reviewer Guide 🔍

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 PR contains tests

🔒 No security concerns identified

⚡ Key issues to review Performance Concern The new implementation fetches all environments, secrets, and variables for each project, which could be inefficient for projects with many items. Consider using aggregation queries or optimizing the database calls. Error Handling The new code doesn't include error handling for database queries. Consider adding try-catch blocks or error handling mechanisms to manage potential failures gracefully.

[codiumai-pr-agent-free[bot]]

PR Code Suggestions ✨

Category	Suggestion	Score
Performance	Optimize database queries by fetching all required counts in a single query ___ **Consider using a single database query to fetch all required counts instead of multiple separate queries for each environment.** [apps/api/src/project/service/project.service.ts [818-863]](https://github.com/keyshade-xyz/keyshade/pull/434/files#diff-d85f483eff15c28a4b3e5f7a54c9b159c71003f8a95ef7a4be00eb80ec72b2faR818-R863) ```diff -const allEnvs = await this.prisma.environment.findMany({ - where: { projectId: project.id } +const projectCounts = await this.prisma.environment.findMany({ + where: { projectId: project.id }, + select: { + id: true, + slug: true, + _count: { + select: { + secrets: { + where: { projectId: project.id } + }, + variables: { + where: { projectId: project.id } + } + } + } + } }) -const envPromises = allEnvs.map(async (env) => { +for (const env of projectCounts) { const hasRequiredPermission = await this.authorityCheckerService.checkAuthorityOverEnvironment({ userId: user.id, entity: { slug: env.slug }, authorities: [Authority.READ_ENVIRONMENT], prisma: this.prisma }) if (hasRequiredPermission) { totalEnvironmentsOfProject += 1 + totalSecretsOfProject += env._count.secrets + totalVariablesOfProject += env._count.variables + } +} - const [secretCount, variableCount] = await Promise.all([ - this.prisma.secret.count({ - where: { - projectId: project.id, - versions: { some: { environmentId: env.id } } - } - }), - this.prisma.variable.count({ - where: { - projectId: project.id, - versions: { some: { environmentId: env.id } } - } - }) - ]) - - totalSecretsOfProject += secretCount - totalVariablesOfProject += variableCount - } -}) - -await Promise.all(envPromises) - ``` - [ ] **Apply this suggestion** Suggestion importance[1-10]: 9 Why: This suggestion offers a substantial performance improvement by reducing the number of database queries, which can lead to faster execution and reduced load on the database.	9
	✅ Optimize performance by parallelizing the fetching of environment, secret, and variable counts ___ **Consider using Promise.all() to parallelize the fetching of environment, secret, and variable counts for better performance.** [apps/api/src/project/service/project.service.ts [822-863]](https://github.com/keyshade-xyz/keyshade/pull/434/files#diff-d85f483eff15c28a4b3e5f7a54c9b159c71003f8a95ef7a4be00eb80ec72b2faR822-R863) ```diff const envPromises = allEnvs.map(async (env) => { const hasRequiredPermission = await this.authorityCheckerService.checkAuthorityOverEnvironment({ userId: user.id, entity: { slug: env.slug }, authorities: [Authority.READ_ENVIRONMENT], prisma: this.prisma }) if (hasRequiredPermission) { totalEnvironmentsOfProject += 1 - - const [secretCount, variableCount] = await Promise.all([ + return Promise.all([ this.prisma.secret.count({ where: { projectId: project.id, versions: { some: { environmentId: env.id } } } }), this.prisma.variable.count({ where: { projectId: project.id, versions: { some: { environmentId: env.id } } } }) ]) - - totalSecretsOfProject += secretCount - totalVariablesOfProject += variableCount } + return [0, 0] }) -await Promise.all(envPromises) +const counts = await Promise.all(envPromises) +totalSecretsOfProject = counts.reduce((sum, [secretCount]) => sum + secretCount, 0) +totalVariablesOfProject = counts.reduce((sum, [, variableCount]) => sum + variableCount, 0) ``` `[Suggestion has been applied]` Suggestion importance[1-10]: 8 Why: The suggestion correctly identifies an opportunity to optimize performance by parallelizing asynchronous operations, which can significantly reduce execution time in scenarios with multiple environments.	8
Enhancement	✅ Enhance test assertions to verify both count and content of fetched project data ___ **Consider adding assertions to verify the actual content of the fetched items, not just their counts.** [apps/api/src/project/project.e2e.spec.ts [604-610]](https://github.com/keyshade-xyz/keyshade/pull/434/files#diff-f1b22baab0b4173d825570c9030ef014582bc75bb67ef2e544f0dd879ff941a2R604-R610) ```diff expect(response.statusCode).toBe(200) expect(response.json().items.length).toEqual(1) -//One environment is added by default -expect(response.json().items[0].totalEnvironmentsOfProject).toEqual(2) -expect(response.json().items[0].totalVariablesOfProject).toEqual(2) -expect(response.json().items[0].totalSecretsOfProject).toEqual(2) +const project = response.json().items[0] +expect(project.totalEnvironmentsOfProject).toEqual(2) +expect(project.totalVariablesOfProject).toEqual(2) +expect(project.totalSecretsOfProject).toEqual(2) +// Verify project details +expect(project.name).toEqual('Project4') +expect(project.description).toEqual('Project for testing if all environments,secrets and keys are being fetched or not') + +// Verify that sensitive data is not included +expect(project).not.toHaveProperty('privateKey') +expect(project).not.toHaveProperty('publicKey') + ``` `[Suggestion has been applied]` Suggestion importance[1-10]: 7 Why: This suggestion enhances the robustness of the test by verifying the content of the fetched data, ensuring that the test checks more than just the counts, which improves test coverage.	7
Best practice	✅ Improve test case naming for better clarity and understanding ___ **Consider using a more descriptive name for the test case to clearly indicate what is being tested.** [apps/api/src/project/project.e2e.spec.ts [530]](https://github.com/keyshade-xyz/keyshade/pull/434/files#diff-f1b22baab0b4173d825570c9030ef014582bc75bb67ef2e544f0dd879ff941a2R530-R530) ```diff -it('should be able fetch all environments,variables,secrets of all projects of a workspace', async () => { +it('should fetch correct counts of environments, variables, and secrets for projects in a workspace', async () => { ``` `[Suggestion has been applied]` Suggestion importance[1-10]: 5 Why: The suggestion improves code readability and maintainability by making the test case name more descriptive, though it does not address a critical issue.	5

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 87.93%. Comparing base (ce50743) to head (d32f207). Report is 150 commits behind head on develop.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## develop #434 +/- ## =========================================== - Coverage 91.71% 87.93% -3.78% =========================================== Files 111 105 -6 Lines 2510 2719 +209 Branches 469 412 -57 =========================================== + Hits 2302 2391 +89 - Misses 208 328 +120 ``` | [Flag](https://app.codecov.io/gh/keyshade-xyz/keyshade/pull/434/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keyshade-xyz) | Coverage Δ | | |---|---|---| | [api-e2e-tests](https://app.codecov.io/gh/keyshade-xyz/keyshade/pull/434/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keyshade-xyz) | `87.93% <100.00%> (-3.78%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keyshade-xyz#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[rajdip-b]

Hold up a min, I'm making some changes.

[rajdip-b]

I have updated the method by which projects are fetched. Doesn't directly affect your changes. Please make the required changes now.