InterviewReady / joy-of-system-design

An online game to kindle the spark of system design in you.
MIT License
153 stars 19 forks source link

Setting up a monorepo and initial project structure. #1

Closed adxy closed 1 year ago

adxy commented 1 year ago

I see that this project requires both frontend & a backend.

A good choice of frameworks would be Node.js (BE), Next.js(FE) & Turborepo to create a monorepo, considering these are the most popular and reliable it would help in getting most contributions while being reliable.

Let me know what you think. If we're on the same page I can start with this.

@coding-parrot

coding-parrot commented 1 year ago

Good point @adxy . I first thought we can skip the backend work by having everything in files and host them on S3 (accessible on CDN).

But looking at it long term, a DB will make things more robust.

Let's start simple. A simple DB with all of it's contents cached in memory sounds good.

Let me consult with a couple of folks and get back to you on the technology. You are right: Simple and popular is better!

adxy commented 1 year ago

Yes, atlas free plan for mongodb would be more than sufficient for the start.

coding-parrot commented 1 year ago

Okay, I think these are the APIs:

add(Graph) -> String graphID getAllGraphs() -> String[] graphIDs get(graphId) -> Graph g validate(Graph) -> (Boolean correct, Map<Vertex, Explanation> explanations)

What about the design though? Should we do something on figma ourselves?

Any designers who can help us here?

adxy commented 1 year ago

Well, let's wait couple of days if someone shows up for UI/UX otherwise we can start with some basic designs. I've some photoshop experience but not much with UI.

I have done UI and everything else for this chesskhelo.in

suyash-purwar commented 1 year ago

I'd love to contribute towards the backend design. For type safety and other features, I think Typescript would be a good choice.

adxy commented 1 year ago

I'd love to contribute towards the backend design. For type safety and other features, I think Typescript would be a good choice.

Personally, I believe, TS is for really large & sophisticated projects. It reduces the development speed and is not really beginner friendly. I would recommend going plain Next.js without TS. Any suggestions?

coding-parrot commented 1 year ago

I would also recommend using a simple and popular language for starting.

Java, Golang, Python, Node.js are the candidates. Taking a poll here: https://app.sli.do/event/oDsZKNQmavYJwc8aow4SVs

adxy commented 1 year ago

Rather than taking a poll on a website where people can just open incognito to vote again. We can do it here!

React with your preferred emoticon in this comment to vote for the backend language.

👍 - Node.js 🚀 - Java ❤️ - Python 👀 - Go

You can even select more than 1 language.

theharshpat commented 1 year ago

NodeJS or Go works, although I would prefer NodeJS. Looks like someone blasted the poll with "java" huh

suyash-purwar commented 1 year ago

I think Node.js would be better as it's more widely used. It would allow more people to contribute.

coding-parrot commented 1 year ago

Sorted, we will use Node.js for the backend!

Where would we host this code? AWS serverless?

adxy commented 1 year ago

Sorted, we will use Node.js for the backend!

Where would we host this code? AWS serverless?

Going serverless (I'm guessing AWS Lambda +AWS API Gateway) for a small service that does 1 task is fine, but here putting the whole BE on serverless would mean migration issues in future if we ever want to move somewhere else.

What about spinning up a basic EC2?

There's also an option with AWS Elastic Bean stack. (Never used this in production.)

We can do manual deployments initially, and make a CI/CD pipeline later on.

Let me know if anyone has a better idea.

theharshpat commented 1 year ago

We can do manual deployments initially, and make a CI/CD pipeline later on.

Let me know if anyone has a better idea.

Digitalocene's droplets VM is also a good way. For keeping the services up and manual deployment we can use pm2, which works very well with node. I have used this in prod.

coding-parrot commented 1 year ago

I think most of our operations on the backend are simple, and just need a CRUD service. A lambda looks good to start with.

EC2 is also fine, I have used both lambda and EC2 on prod. We used to use PM2 for health checks as @Magic105 mentioned.

Vote: ❤️ - EC2 🚀 - Lambda

nivendha commented 1 year ago

if I may, please add firebase to the pool, it would be a go-to option for the initial product phase, and since it's going to be over the GCS providers if the day comes you want to get more features to scale, it shouldn't be an issue,

PS: I am not selling firebase :P I love all providers equally( except azure) Reasons to Consider would be

  1. You don't have to spend time setting up any of your service infras
  2. They provide multiple Authentication provider wrappers for a quick start, password and SMTP
  3. You get to have hoisting (also subdomains like dev and test can be done), serverless functions (node js) and Analytics integration
  4. They provide a local playground for you to test your functions and code before deployment
  5. U can forget a server/service for CI/CD and make use of Firebase GitHub actions
  6. Use free tire, its really good

Challenges you will face

  1. If you are going with Next js, then you will have a hard time if you want server-side rendering on any of your pages( these have to tweak a bit to get the rendering done via serverless function, its possible, but a bit of effort needed), my suggestion go with client-side/static generation PS: Don't fall for the latest version of firebase-CLI which allows the Next js starter template, last time I checked there are still some issues in the deployment via that 2 Please be careful using the Firestore(no SQL), in the way you design and update your data model, the charge for read /writes and a root/parent updation is going to be costly - just a heads up u might know this point PS: I usually try to have one Droplet in the digital ocean and deploy Hasura - a cheaper instance is okay and easy to develop - Plz take backups at regular time

Not relevant/important, but consider these if it's okay

  1. Please have unit testing added to the CI/CD, if devs are busy, make this a good first issue for getting more traction to contribute, saying this bcs thing usually scale faster and when multiple PR comes, devs might not be able to deep review any of the code
  2. A simple page design with just text questions and using react-flow to play and answer is something I love, and hope most people would too
esterkaufman commented 1 year ago

I'm following this thread because the idea described in the ReadMe file, can be very significant for many, many people. So, if you guys will need nodeJs contributors, please reach out.

adxy commented 1 year ago

if I may, please add firebase to the pool, it would be a go-to option for the initial product phase, and since it's going to be over the GCS providers if the day comes you want to get more features to scale, it shouldn't be an issue,

PS: I am not selling firebase :P I love all providers equally( except azure) Reasons to Consider would be

  1. You don't have to spend time setting up any of your service infras
  2. They provide multiple Authentication provider wrappers for a quick start, password and SMTP
  3. You get to have hoisting (also subdomains like dev and test can be done), serverless functions (node js) and Analytics integration
  4. They provide a local playground for you to test your functions and code before deployment
  5. U can forget a server/service for CI/CD and make use of Firebase GitHub actions
  6. Use free tire, its really good

Challenges you will face

  1. If you are going with Next js, then you will have a hard time if you want server-side rendering on any of your pages( these have to tweak a bit to get the rendering done via serverless function, its possible, but a bit of effort needed), my suggestion go with client-side/static generation PS: Don't fall for the latest version of firebase-CLI which allows the Next js starter template, last time I checked there are still some issues in the deployment via that 2 Please be careful using the Firestore(no SQL), in the way you design and update your data model, the charge for read /writes and a root/parent updation is going to be costly - just a heads up u might know this point PS: I usually try to have one Droplet in the digital ocean and deploy Hasura - a cheaper instance is okay and easy to develop - Plz take backups at regular time

Not relevant/important, but consider these if it's okay

  1. Please have unit testing added to the CI/CD, if devs are busy, make this a good first issue for getting more traction to contribute, saying this bcs thing usually scale faster and when multiple PR comes, devs might not be able to deep review any of the code
  2. A simple page design with just text questions and using react-flow to play and answer is something I love, and hope most people would too

About hosting, please feel free to correct me if I am wrong but we are deciding where to host backend and not frontend.

The FE Next.js application can be deployed on Vercel with automatic CI/CD, Cache, SSR and gives an option for multiple environments like staging and production with custom domains while being 100% free for us.

nivendha commented 1 year ago

True, agreed Vercel is the god for now when it comes to next js, it has all the features, But I don't like it only for the reason that, when you want to scale and bring more collaborators in forming a team, you will be paying for each member. I felt that was burning a hole in my pocket If you are confirmed to choose node js (asynchronous event-driven) I would think of not spending on Instances which are always up, rather going serverless PS: cost is a heavy point I am pointing out, agreeably it might not matter now

coding-parrot commented 1 year ago

Folks let's continue the conversation here: https://join.slack.com/t/joyofsystemde-dnx5265/shared_invite/zt-1n7pnkbc2-aP_2gJ1AhqS9aLvtc9WTnw