search

CS3219-AY2223S1 / cs3219-project-ay2223s1-g42

Collaborative peer code interviewing web app project for CS3219
https://cs3219-g42.vercel.app
4 stars 3 forks source link

[Docs] Project Requirements #3

Open yusufaine opened 2 years ago

yusufaine commented 2 years ago

THIS ISSUE SERVES AS A DRAFT FOR PROJECT REPORT.

Decisions

Tech stack

Deployment

For the development and demonstration of the application, we would focus on running the service locally. Once all the features are up and running nearer and as we approach the deadline, we would ideally dockerise some of the microservices and deploy it as follows:

Frontend

Vercel Google App Engine
Price Free for personal repositories. Free for the first 28 instance hours of "F" instances
Availability Available 24/7, also provides readable URLs when deployed Available 24/7, default URLs are not very readable
Scaling Unable to find, but should not be an issue Handled by Google for free, within the deployed instance class (F1 - F4), different increasing memory and CPU limits.
CI/CD Difficulty Vercel has their own CICD pipeline To find out more on, but currently thinking of manually integration and deployment

Leaning towards Vercel if we can integrate it with our backend.

Backend

Google Cloud Run Google App Engine
Price Free 180000 vCPU seconds per month Free for the first 28 instance hours of "F" instances
Availability Pay-per-use, first request would is slow as it needs to "spin up" Available 24/7, does not require "spinning up"
Scaling Handled by Google for free Handled by Google for free, within the deployed instance class (F1 - F4), different increasing memory and CPU limits.
CI/CD Difficulty Relatively simple, can be done with Github Actions (example) To find out more on, but currently thinking of manually integration and deployment

Leaning towards Google App Engine.

User service

Planetscale -- Hosted SQL database.

  1. Reasonable free tier
  2. Chosen over NoSQL service as the data in our service can be relational and we would like the ability to work with that.

Matching service

Utilise Redis' namespace as the different types of "queues" (PubSub topics).

Question service

Planetscale to allow for relational queries/mappings, Redis for quick queries.

Design

We will prioritise functionality first before user experience. Simply put, the team would work on getting a functional version up first before improving on the UI elements that would improve the user experience.

Diagrams

All of our diagrams are stored in this Gist.

Architecture

Simplified

image

Specifics

User flow

image

Specifics

  1. At any page after Dashboard users can navigate access the pages that are immediately a child to Dashboard as there would be a navigation bar.
    • Locked room page with past session can access Matching page directly without needing to go back to Dashboard.
  2. At any page, a user may choose to log out and would be directed back to the start of the user flow.
yusufaine commented 2 years ago

User Service

Authentication

Functional Requirements

  1. Allow users to create an account with username and password.
  2. Ensure that every account created has a unique username.
  3. Allow users to sign in by entering their username and password.
  4. Allow users to sign out of their account.
  5. Allow users to delete their own account.
  6. Allow users to change their password.
  7. Allow users to reset their password.

Non-functional Requirements

  1. Write operations of an account should only be able to be carried out by the user themselves, or the admin
  2. Read operations of an account should only display the relevant information
    • Users should not be able to see each other's passwords
    • Users should be allowed to see other non-private fields related to the service (historical data/sessions) -- low priority, depends on deployment of History Service
  3. Resetting of account should only be done to the appropriate user.
    • If Alice resets the password, only Alice should be allowed to set the new password.
  4. User service should be able to support at least 100 users.
  5. Users passwords should be hashed and salted before storing in the database.

Progress

History (Stretch Goal)

Functional Requirements

Non-functional Requirements

Progress

Diagrams

Authentication

user-authentication

History

user-history

yusufaine commented 2 years ago

Matching Service

Functional Requirements

  1. Allow users to select the difficulty level of the questions they wish to attempt.
  2. Match 2 waiting users with similar difficulty levels and put them in the same room.
  3. System should pair matching users within 30 seconds. (Pair within 30 seconds)
  4. Inform users that no match is available if a match cannot be found within 30 seconds.
  5. Users should be able to explicitly leave a room once matched.

Non-functional Requirements

  1. Matching service should allow at least 100 concurrent users to be in the queue

Progress

Stretch Goals

Diagram(s)

Finding a match

image

No match found, joining the queue

image

Match found, joining room

image

User is already in a room

image

yusufaine commented 2 years ago

Question Service

The service is linked to backend and utilises the following:

  1. PlanetScale -- Hosted MySQL database (also used by User Service)
  2. Prisma -- ORM for PlanetScale
  3. Firebase/GCP Cloud Functions
    • HTTP-trigger to manually seed,
    • Event-triggered (PubSub + Cloud Scheduler) to simulate cron job.

Functional Requirements

  1. Matched users should be served the same question based on their matching criteria.
  2. System should be able to get questions by difficulty and topic.
  3. Users should be able to see the latest details of the question at a glance.

Non-functional Requirements

  1. Questions, summary and details, should be served to users within 3 seconds.

Detailed Design

Addressing FR and NFR

  1. Store the question details in a database -- resolve availabilty issues.
    • This is to prevent the service from being strongly coupled to (depend on) Leetcode's endpoint.
    • If Leetcode becomes unavailable -- goes down, blocks public access to data -- or changes the endpoint, the service would not be able to serve the users anything.
    • The only time the service would not be able to serve questions is if the database goes down.
      • Nice-to-have: Automate backups in the event the database crashes and loses all data. Handled by PlanetScale, free daily backup. :+1:
  2. Users will be served updated data as the database gets updated every hour and checks for

Potential Issues

  1. Issue: Exceeding runtime of Cloud Function
    • Cloud Functions runtime is limited to 540 seconds (see here), and Prisma does not support createMany and establish relations, this has to be done one-by-one.
    • There is no known Leetcode GraphQL endpoint that returns the QuestionContent of all questions and it has to be done manually.
    • With ~2000 free Leetcode questions and querying each content takes ~300ms, querying all the questions would take ~600 seconds, prior to accounting for updating QuestionSummary, retries and writing to the database and would overrun the limitation of Cloud Function.

To resolve this:

  • Separate updating the QuestionSummary and QuestionContent, and execute them sequentially in that order (content relies on summary).
  1. Issue: Leetcode rate-limiting queries and potential network I/O errors
    • As mentioned in the point above, at worst, the service would need to query all question contents. This does happen when the database is fresh or if LeetCode adds a lot of new questions at once.

To resolve this:

  • Process LC queries by chunks rather than sending all the requests at once (Promise.all())
    • Seeding the database: This does not fully resolve the issue but yields better results (more HTTP status 200)
    • Updating the database: This should not prove to be an issue as it is unlikely that LC adds >10 questions in a day.

The following were implemented but only (2) is deployed:

  1. Creating a HTTP-triggered function and repeat it until there are no leftovers, or
  2. Increase the frequency of the event-triggered function.

The service implements both to seed and update by the hour every hour to update the summary values, daily quetion and resolve and leftover missing content, if any.

Schemas

Updated as of: 25th Septemeber 2022

This is based of what LeetCode returns, for the actual schema that was applied, refer to prisma/schema.prisma which also covers the relations such as the one-to-many relation that was mentioned in the first point of the previous section.

QuestionSummary
Column Name Data Type Description
acRate number Solution acceptance rate (%)
difficulty string Difficulty level of the question
paidOnly boolean Is the question only available to paid users?
title string Question title
titleSlug string Slug associated to question
topicTags string[] Topics that are associated to the question
createdAt timestamp When the question was seeded
updatedAt timestamp When the question was last updated
QuestionContent
Column Name Data Type Description
content string Question context and details.
hints string[] Hints to solving the question, if any.
topicTags string[] Topics that are associated to the question
discussionLink string Link to the question's discussion page on Leetcode
createdAt timestamp When the question was seeded
updatedAt timestamp When the question was last updated

GraphQL Queries

Updated as of: 25th Septemeber 2022

All questions
query psetQuestionList {
  psetQuestionList: questionList(
    categorySlug: ""
    limit: -1
    skip: 0
    filters: {}
  ) {
    total: totalNum
    questions: data {
      acRate
      difficulty
      paidOnly: isPaidOnly
      title
      titleSlug
      topicTags {
        slug
      }
    }
  }
}
Question of the day
query questionOfToday {
  activeDailyCodingChallengeQuestion {
    question {
      acRate
      difficulty
      paidOnly: isPaidOnly
      title
      titleSlug
      topicTags {
        slug
      }
      hasSolution
      hasVideoSolution
    }
  }
}
Get question from slug
query getQuestionDetail($titleSlug: String!) {
  question(titleSlug: $titleSlug) {
    content
    hints
    topicTags {
      name
    }
  }
}

# var
# {
#  "titleSlug": "two-sum"
# }

Potential upgrades

Progress

Diagram

question-service

yusufaine commented 2 years ago

COLLABORATION SERVICE MERGED UNDER ROOM SERVICE

Hidden
## Collaboration Service ### Functional Requirements * Users would need to be able to live-edit a text document and changes should be synchronised across both parties. ### Non-functional Requirements * Latency should be < 1000 milliseconds. ### Progress - [x] Working collaborative editor (Week 7)
yusufaine commented 2 years ago

User Interface

Design Decisions

We prioritise XYZ and as such did the following:

  1. X
  2. Y
  3. Z
Hidden
### Functional Requirements ### Non-functional Requirements ### Progress - [ ] TO UPDATE, IDEALLY LINK TO PR USING `[text](link_to_pr)` - [ ] AND MARK UPDATED ACCORDINGLY
yusufaine commented 2 years ago

COMMUNICATION SERVICE MERGED UNDER ROOM SERVICE

Hidden
## Communication Service ### Functional Requirements * Users should be able to orally communicate with their match. ### Non-functional Requirements * Communication needs to be as close to real time as possible. ### Progress - [ ] Voice and video data through websockets - [ ] UI
yusufaine commented 2 years ago

HISTORY SERVICE MERGED UNDER USER SERVICE

Hidden
## Learning Pathway/History Service ### Functional Requirements * Users may want to view they past attempts to see how they did it before and compare if they've made any improvements. ### Non-functional Requirements * System should be able to get all of the user's past historical data since feature inception. ### Progress Backend: * [ ] Add table to DB associating attempts to users, should only contain userID, title slug, submission attempt, and when it was created at * [ ] API endpoint to GET all submissions ("summary"), get contents of specific submission, and POST to add new submissions Frontend: * [ ] "Submit" button in room service
yusufaine commented 2 years ago

Room Service

Room

Rooms serve the purpose of containing the question content from the question service, and host the collaboration and communication service.

Functional Requirements

  1. Users should be able to explicitly leave the room.
  2. Matched users should have the same questions synced in their view.

Non-functional requirements

Stretch goal

Collaboration Service

This service mainly covers the ability to work collaboratively on the same document as the partner's.

Functional Requirements

  1. Changes made by to the document by one user needs to be reflected on the other's.
  2. The shared document needs to persist in cases of leaving the room accidentally.
  3. Document needs to be cleared when users explicitly disconnect/leave.

Non-functional Requirements

  1. The system needs to be as close to real time as possible (<1000 milliseconds).

Communication Service

While working on the same document, users would need a way to communicate their ideas/suggestions across to each other. Collaboration service can function in place of a chat service through comment blocks but does not simulate a proper interview/discussion and is no where near as interactive.

Functional Requirements

  1. Users should be able to orally communicate with their match.

Non-functional Requirements

  1. Communication needs to be as close to real time as possible

Progress