search

PalisadoesFoundation / talawa-api

API Backend for the Talawa Mobile App. Click on the link below to see our documentation
https://docs.talawa.io/
GNU General Public License v3.0
224 stars 816 forks source link

Feature: Added MinIO Setup and Configuration for Local and Docker Environments (GSoC'24) #2476

Closed chandel-aman closed 1 month ago

chandel-aman commented 2 months ago

What kind of change does this PR introduce?

Feature: Adds MinIO setup and configuration instructions for both local and Docker environments.

Issue Number:

Fixes #2419

Did you add tests for your changes?

Yes

Snapshots/Videos:

Added MinIO setup

https://github.com/user-attachments/assets/ff8783ae-9053-4732-8cea-1b9b4c8cfe5c

MinIO docker setup

https://github.com/user-attachments/assets/0007cad5-014b-48e0-aa3e-818185852f0c

MinIO local setup

https://github.com/user-attachments/assets/0fe0f143-b8bd-4b7e-9db3-4855a5ff0e6e

If relevant, did you update the documentation?

Yes

Summary

This PR introduces MinIO setup instructions into the project to streamline its deployment and configuration. Specifically, it adds:

Have you read the contributing guide?

Yes

Summary by CodeRabbit

coderabbitai[bot] commented 2 months ago

Walkthrough

The changes integrate MinIO, an object storage service, into the project. This includes the introduction of an asynchronous function for configuring MinIO settings based on user input, enhancements to npm scripts for checking and running MinIO, and updates to the environment configurations. Additionally, the setup process is improved by prompting users for necessary credentials and ensuring the environment is correctly set up for MinIO usage.

Changes

Files Change Summary
.eslintrc.json, setup.ts, package.json Updated ESLint configuration to include setup.ts, added a new asynchronous function configureMinio for configuring MinIO settings, and introduced new npm scripts for MinIO integration.

Assessment against linked issues

Objective Addressed Explanation
Docker Integration for MinIO ( #2419 ) ❓ The PR does not explicitly detail Docker configurations.
npm Script Integration for starting MinIO with API server ( #2419 ) ✅
Update project documentation for MinIO usage ( #2419 ) ❌ Documentation updates are not included in the PR.
Ensure MinIO installation checks ( #2419 ) ✅

🐇 In fields of green, a treasure we find,
MinIO joins us, storage well-defined.
Docker and scripts make setup a breeze,
Hopping along with data's new ease!
So let’s celebrate with a joyful cheer,
Our files secure, with MinIO near! 🥕✨

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share - [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)
Tips ### Chat There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai): - Review comments: Directly reply to a review comment made by CodeRabbit. Example: -- `I pushed a fix in commit , please review it.` -- `Generate unit testing code for this file.` - `Open a follow-up GitHub issue for this discussion.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: -- `@coderabbitai generate unit testing code for this file.` -- `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: -- `@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.` -- `@coderabbitai read src/utils.ts and generate unit testing code.` -- `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` -- `@coderabbitai help me debug CodeRabbit configuration file.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (Invoked using PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai full review` to do a full review from scratch and review all the files again. - `@coderabbitai summary` to regenerate the summary of the PR. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository. - `@coderabbitai help` to get help. ### Other keywords and placeholders - Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. - Add `@coderabbitai summary` to generate the high-level summary at a specific location in the PR description. - Add `@coderabbitai` anywhere in the PR title to generate the title automatically. ### Documentation and Community - Visit our [Documentation](https://coderabbit.ai/docs) for detailed information on how to use CodeRabbit. - Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback. - Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.
github-actions[bot] commented 2 months ago

Our Pull Request Approval Process

We have these basic policies to make the approval process smoother for our volunteer team.

Testing Your Code

Please make sure your code passes all tests. Our test code coverage system will fail if these conditions occur:

  1. The overall code coverage drops below the target threshold of the repository
  2. Any file in the pull request has code coverage levels below the repository threshold
  3. Merge conflicts

The process helps maintain the overall reliability of the code base and is a prerequisite for getting your PR approved. Assigned reviewers regularly review the PR queue and tend to focus on PRs that are passing.

Reviewers

Do not assign reviewers. Our Queue Monitors will review your PR and assign them. When your PR has been assigned reviewers contact them to get your code reviewed and approved via:

  1. comments in this PR or
  2. our slack channel

Reviewing Your Code

Your reviewer(s) will have the following roles:

  1. arbitrators of future discussions with other contributors about the validity of your changes
  2. point of contact for evaluating the validity of your work
  3. person who verifies matching issues by others that should be closed.
  4. person who gives general guidance in fixing your tests

CONTRIBUTING.md

Read our CONTRIBUTING.md file. Most importantly:

  1. PRs with issues not assigned to you will be closed by the reviewer
  2. Fix the first comment in the PR so that each issue listed automatically closes

Other

  1. :dart: Please be considerate of our volunteers' time. Contacting the person who assigned the reviewers is not advised unless they ask for your input. Do not @ the person who did the assignment otherwise.
  2. Read the CONTRIBUTING.md file make
codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 73.09417% with 60 lines in your changes missing coverage. Please review.

Project coverage is 98.77%. Comparing base (c952c7a) to head (c177553). Report is 1 commits behind head on develop.

Files with missing lines Patch % Lines
src/minioInstallationCheck.ts 0.00% 60 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #2476 +/- ## =========================================== - Coverage 99.09% 98.77% -0.33% =========================================== Files 349 355 +6 Lines 17769 17992 +223 Branches 2371 2400 +29 =========================================== + Hits 17609 17772 +163 - Misses 160 220 +60 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

chandel-aman commented 2 months ago

@palisadoes @aashimawadhwa Could you please review this PR?

palisadoes commented 2 months ago

@aashimawadhwa @tasneemkoushar PTAL

palisadoes commented 2 months ago

Please also work on the code rabbit suggestions for improving the code base

tasneemkoushar commented 2 months ago

@chandel-aman did you make any fix for the error that @palisadoes pointed to? I tried doing the setup but didn't get that error in my case image

tasneemkoushar commented 2 months ago

@palisadoes works fine with sample imports as well image

tasneemkoushar commented 2 months ago

@chandel-aman while docker setup step, i got this error response, is this anyhow related to your change image

tasneemkoushar commented 2 months ago

@chandel-aman Tested minio local setup, works fine

chandel-aman commented 2 months ago

  1. I get this error when trying it.

    1. I was evaluating a PR before that made some DB updates as part of the setup process. This may have been the cause.

      1. Make sure that the setup script drops the entire database before loading data in both the default and sample data loading steps.

    2. These were the values I used in the setup. You can see whether this was the cause too.

MINIO_ENDPOINT=http://localhost:9000
MINIO_ROOT_USER=peter
MINIO_ROOT_PASSWORD=peter-talawa
MINIO_BUCKET=peter-bucket

image

image

The error you're experiencing is not related to the Minio setup or the environment variables you shared. It's a MongoDB duplicate key error in the talawa.api.users collection.

The error occurs because of duplicate null values in the "identifier" field, which has a unique index. This is likely due to the previous database state rather than the Minio configuration.

chandel-aman commented 2 months ago

I've made changes to drop the database before loading data in both default and sample data loading steps.

chandel-aman commented 2 months ago

  1. The setup script doesn't check to see whether MinIO is installed. I ran the setup script successfully using the default data and there was no error.

    1. Please make sure the appropriate checks are made

    2. Please work with you mentors on the docker setup verification

Yes, you're correct that the setup script currently does not check for MinIO installation. Instead, the check and installation are performed when the user runs the command npm run <dev or start>:with-minio. Would you like me to include the check and installation for MinIO in the initial setup script itself?

tasneemkoushar commented 2 months ago

@chandel-aman including a check would work and can you please check the lining or some formatting issue in your commit.

chandel-aman commented 2 months ago

@chandel-aman did you make any fix for the error that @palisadoes pointed to? I tried doing the setup but didn't get that error in my case image

No, I did not make any fixes for the error pointed by @palisadoes. The error is not caused by the changes in this PR but is likely due to existing changes made to the database.

chandel-aman commented 2 months ago

@chandel-aman while docker setup step, i got this error response, is this anyhow related to your change image

No, this error is not related to my changes. It might be occurring due to this line: ${PWD}/Caddyfile:/etc/caddy/Caddyfile. It could be either a path error or a permission error. I've verified that the Docker setup is running correctly. image

chandel-aman commented 2 months ago

@chandel-aman including a check would work and can you please check the lining or some formatting issue in your commit.

Sure, I'll make the changes to include a check for Minio in the setup.

I’ve fixed the linting error; however, other linting issues are occurring due to the @typescript-eslint/ban-types rule, which has been deprecated in version 8. You can find more information here. This might be related to the PR #2485.

palisadoes commented 2 months ago

I’ve fixed the linting error; however, other linting issues are occurring due to the @typescript-eslint/ban-types rule, which has been deprecated in version 8. You can find more information here. This might be related to the PR #2485.

Please create an issue to fix this so that we have better code quality. Hopefully someone else will give it a try.

tasneemkoushar commented 2 months ago

@chandel-aman do let me know when you PR is ready for merging

palisadoes commented 2 months ago
  1. Please make sure that the instructions in INSTALLATION.md are clear and flow logically. For example, we now have a new prerequisite to make the application work.
  2. Also, how will the user be able to change the location of the data/ directory? This should be documented and possibly given as a setup option.
  3. If all objects are going to be uploaded to the data/ directory, is the images/ directory needed in the repo?
  4. If the data/ directory is required, then it should be added to the repo. Take a look at what was done for the images/ directory, both in its content and its treatment in the .gitignore file
palisadoes commented 2 months ago

Please make the coderabbit.ai edits

chandel-aman commented 2 months ago

  1. Please make sure that the instructions in INSTALLATION.md are clear and flow logically. For example, we now have a new prerequisite to make the application work.

    1. Also, how will the user be able to change the location of the data/ directory? This should be documented and possibly given as a setup option.

    2. If all objects are going to be uploaded to the data/ directory, is the images/ directory needed in the repo?

    3. If the data/ directory is required, then it should be added to the repo. Take a look at what was done for the images/ directory, both in its content and its treatment in the .gitignore file

  1. I've made the data/ directory configurable so as to allow user to change the location of this directory.
  2. No, we won't we needing the images/ or videos/ directory, and hence we can remove them from the repo.
  3. No, the data/ directory is not required to be present in the repo. I did look into the earlier configuration, we are not including any directories which stored files.
chandel-aman commented 2 months ago

@chandel-aman do let me know when you PR is ready for merging

@tasneemkoushar @aashimawadhwa Could you please review this PR?

aashimawadhwa commented 2 months ago

Hey @chandel-aman this looks good to me i have tried running this on my local is working good @palisadoes we can merge it, it's gtg.

chandel-aman commented 2 months ago

@palisadoes @tasneemkoushar Could you please provide your review?

palisadoes commented 2 months ago

Please fix the failing tests

chandel-aman commented 1 month ago

Please fix the failing tests

The failing test has been fixed. Could you please merge this PR?

aashimawadhwa commented 1 month ago

@palisadoes this is gtg