[DMP 2024]: Indian Version of QuoDB Site

[arvind-planetread]

Ticket Contents QuoDB is an online website that allows users to search for movie quotes and show results across multiple categories. We want to create a similar site which will work for Indian Language movies. Especially for Hindi, Tamil and Telugu. This will easily get the interest of so many millions of movie fans of Bollywood, Tollywood and Kollywood in India and the world. The starting figure itself could be easily 50 Million Plus.

Goals & Mid-Point Milestone Goal 1: Create a simple and clean UX and UI for the frontend of the site. Create an admin panel for the site so that we or an authorised user from our side can add movie script information to the site. Support Hindi Language.

Goal 2: Support Tamil and Telugu Language support. Allow sharing of the quotes in various web medium. Make the website fully responsive and mobile friendly. Add accessibility features to search by voice etc.

Goal 1 is the midpoint.

Setup/Installation No response

Expected Outcome The final version of the site should mimic quodb.com for Indian language movies in Hindi, Tamil and Telugu.

Acceptance Criteria The acceptance criteria will be a fluid, functional website which has 90% plus accuracy with search.

Implementation Details We welcome all open tech stack which can support this website which will be scaled to more languages and more content in the future.

Mockups/Wireframes No response

Product Name IndoMovieQuo

Organisation Name Planet Read

Domain ⁠Education

Tech Skills Needed Bootstrap, Database, Design, JavaScript, Mockups, Python, SQL, UI/UX/Design

Mentor(s) @arvind-planetread

Category Beginner Friendly, Database, Frontend, Mobile, Question

[Arbtrage]

Weekly Goals

Week 1

• [x] Researched potential integration strategies for Elasticsearch with the FastAPI backend to optimize search functionalities.
• [x] Explored various embedding models suitable for converting movie quotes into searchable embeddings, including those offered by OpenAI, Mistral, and Sentence Transformers.
  • [x] Conducted an in-depth review of Elasticsearch documentation to understand best practices for building a semantic search engine capable of handling complex queries.

Week 2

• [x] Developed the initial API routes in FastAPI, incorporating basic Docker configurations for container management.
• [x] Established database connections for Elasticsearch and configured Docker containers to ensure seamless service interaction.
• [x] Configured Kibana for real-time monitoring of Elasticsearch indices to facilitate easy debugging and data visualization.

Week 3

• [x] Began integrating Elasticsearch for robust management of movie quotes data, ensuring a scalable setup.
• [x] Designed basic data models in Elasticsearch to efficiently store and retrieve movie quotes along with related metadata.
• [x] Implemented essential CRUD (Create, Read, Update, Delete) operations through FastAPI, enhancing the backend's capability to interact dynamically with Elasticsearch.

Week 4

• [x] Integrated Celery with Redis and Flower to manage long-running jobs effectively, such as bulk data uploading from CSV files.
• [x] Continued enhancement of the API to support dynamic database interactions and ensure data consistency.
• [x] Focused on improving error handling and data validation within the API to prevent data corruption and enhance user trust.
• [x] Encountered and began troubleshooting a serialization issue in the worker threads, affecting the handling of movie quote data.

Week 5

• [x] Directed efforts towards resolving the serialization error in worker threads to ensure smooth data processing.
• [x] Initiated the setup of the Next.js application, focusing on routing and basic framework configuration using Create Next App.
• [x] Developing foundational UI components, including a search bar, quote display cards, and intuitive navigation menus to enhance user interaction.
• [x] Researching further into Elasticsearch capabilities to implement support for multilingual search, accommodating diverse user bases.

Week 6

• [x] Implemented multilingual search functionality using Elasticsearch.
• [x] Integrated CockroachDB to store metadata related to the quotes.
• [x] Developed a mechanism to retrieve quotes along with search results from the Elasticsearch database.

Week 7

• [x] Initiated the deployment of frontend code on Vercel.
• [x] Created configuration files for the monorepo setup.
• [x] Continued work on implementing the Elasticsearch search mechanism.

Week 8

• [x] Conducted testing of the search functionality.
• [x] Created Docker files for containerizing the server build.

Week 9

• [x] Integrated server APIs with the frontend.
• [x] Initiated server deployment on AWS EC2.
• [x] Configured Nginx for public server access.

Week 10

• [x] Tested and optimized the search functionality to improve single result display.
• [x] Implemented bulk upload options using CSV and SRT files.

Week 11

• [x] Conducted thorough testing of the entire application.
• [x] Merged the monorepo into a single repository and deployed both the frontend and backend on a single EC2 instance.

Week 12

• [x] Monitored deployment for bugs and addressed crashes.
• [x] Enhanced site responsiveness.
• [x] Worked on technical and user documentation.