Closed amindadgar closed 4 months ago
[!WARNING]
Rate Limit Exceeded
@amindadgar has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 2 minutes and 58 seconds before requesting another review.
How to resolve this issue?
After the wait time has elapsed, a review can be triggered using the `@coderabbitai review` command as a PR comment. Alternatively, push new commits to this PR. We recommend that you space out your commits to avoid hitting the rate limit.How do rate limits work?
CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our [FAQ](https://coderabbit.ai/docs/faq) for further information.Commits
Files that changed from the base of the PR and between ec9717b49b2d39819c5b0a01badd2837774098e1 and 849278b33869d6646b76872dc59638a6fab91ec4.
The recent updates bring a new qdrant
service to the Docker configurations, replacing the previously commented-out mongo
service. The init.sh
script has been simplified by removing Airflow version checks, and the Dockerfile
now installs a specific version of Airflow. Various ETL-related scripts and tests have been updated to reflect new variable names and import paths for improved clarity and organization.
File Path | Change Summary |
---|---|
docker-compose.test.yml , docker-compose.yaml |
Added qdrant service with configuration, health checks, and dependencies; commented out mongo service in docker-compose.yaml . |
init.sh |
Removed version comparison logic and associated error messages for Airflow. |
Dockerfile |
Changed to install a specific version of Airflow (2.9.1 ) before installing other requirements. |
dags/hivemind_etl_helpers/notion_etl.py , dags/hivemind_etl_helpers/tests/integration/test_ingestion_pipeline_etl.py |
Renamed variable table_name to collection_name in ETL processing and tests. |
dags/hivemind_etl_helpers/tests/integration/test_github_etl_fetch_commits.py |
Updated import path for GithubExtraction . |
dags/hivemind_etl_helpers/tests/unit/test_notion_extractor.py , dags/hivemind_notion_etl.py |
Updated import paths for NotionExtractor and process_notion_etl . |
dags/hivemind_etl_helpers/src/utils/credentials.py |
Added load_qdrant_credentials() function to load and validate Qdrant credentials from environment variables. |
dags/hivemind_etl_helpers/src/utils/qdrant.py |
Introduced QdrantSingleton class to manage a singleton instance of a Qdrant client. |
dags/hivemind_etl_helpers/tests/integration/test_load_envs.py |
Added tests for loading Qdrant credentials and configuring MongoDB credentials. |
🐇 In Docker's realm, a
qdrant
rose, Replacingmongo
in repose. Scripts refined, paths realigned, With Airflow's checks now left behind. A singleton for Qdrant's might, ETL flows in clearer light. 🌟
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?
Summary by CodeRabbit
New Features
qdrant
service for enhanced database capabilities.Bug Fixes
Refactor
table_name
tocollection_name
for consistency in ETL scripts and tests.TestIngestionPipeline
class for improved clarity.Chores
mongo
service in thedocker-compose.yaml
file.