wildan3105 / google-scraper

Google scraper
https://google-scraper-web-7922a635d04a.herokuapp.com/
0 stars 1 forks source link

[Question] Work and effort prioritization #40

Open malparty opened 5 months ago

malparty commented 5 months ago

Could you share more about how you split the work and organize the development?

Besides, what were the drivers for deciding to implement manually the authentication (rather than using existing libs)? (the first PR does not give much information about it)

wildan3105 commented 5 months ago

Flow:

  1. Problem Definition and Requirements:

    • Identified the problem and outlined both functional and non-functional requirements.
    • Made decisions regarding technological choices for the project.
  2. Backend Development:

  1. Core Functionality Implementation: CSV Upload

    • Implemented CSV upload functionality, including validation and parsing.
    • Established a pub/sub mechanism to handle keyword uploads and scraping.
  2. Google Scraping Integration:

    • Extended functionality to include scraping Google search results.
    • Implemented a worker to compress HTML code and store relevant information in the database.
  3. Frontend Setup:

    • Set up initial frontend functionalities mirroring backend features.
    • Included user sign-up, sign-in, CSV upload, and keyword management.
  4. Integration, Testing, and Deployment:

    • Integrated frontend and backend components.
    • Created several tests to cover some areas of the application.
    • Deployed the application to verify functionality in a live environment.

Authentication Flow:

  1. Users sign in with their chosen password during the sign-up process.
  2. Password verification is performed against the encrypted password stored in the database.
  3. Upon successful verification, a token is generated using a specified secret key and algorithm, containing user ID and email for future processes.
  4. The token is provided in the login response for secure API calls.

Reasoning Behind Manual Authentication Implementation:

Let me know if you need more clarification!

malparty commented 5 months ago

Thanks, that's clear enough :)