Define Repository Structure and Tech Stack for Mob Programming Monorepo

[Konosh93]

This issue aims to discuss and define the structure of our monorepo and the technologies to be included in the boilerplate code. To ensure a smooth and efficient mob programming experience, we need to decide on the frontend and backend frameworks, as well as any AI libraries that we'll be using.

Please share your thoughts on the following topics:

Repository structure: How should we organize the frontend, backend, and AI components within the monorepo? Frontend frameworks: What frontend frameworks (e.g., React, Angular, Vue) should we include in the boilerplate? Backend frameworks: What backend frameworks (e.g., Express, Django, Flask) should we incorporate? AI libraries: What AI libraries (e.g., TensorFlow, PyTorch, scikit-learn) are essential for our mob programming projects? Additional considerations: Are there any other tools, libraries, or best practices we should adopt for our monorepo? Let's collaborate to create a solid foundation for our mob programming sessions!

[NikitaKurpas]

Before I reply, for this or next time, we could try using GH Discussions for things that are, well, discussions. They have some features that GH Issues don't, like replies.

---

General setup

• Use VS Code as the editor because 1) we can use dev containers (see below), and 2) we can use Live Share to collaborate.
• Use Live Share to collaborate on the code on just one or a few machines to reduce the friction of integrating each other's code and see results instantly.
• Use dev containers to avoid the need for people to install all dev tools locally. Install scripts in the dev containers will set up the env, install all dependencies, libs, VS Code plugins, config, etc.
  • The only prerequisite is Docker, which, I assume, many people already have.
• Use iPython Notebooks to experiment and try out ideas quickly before coding them. Setting up notebooks in VS Code is pretty easy.

Backend frameworks

Since a lot of AI tools and libs are only available for Python, I think we should go with it. Depending on what we will write, I propose the following:

• If we will write some APIs, we can use the FastAPI framework. It is simple and very easy to get started with. Just check the docs.
• If we will build a web UI, I propose we use Streamlit (see the frontend section below). It is also very easy to get started with.

Frontend frameworks

Since the topic of this collaboration session is AI, I think we should use the simplest and easiest setup for the frontend. Hence, I propose we use either:

• Use Streamlit library for Python - looks very easy to get started with.
• Use React with Remix and Tailwind - IMO it is the easiest to get started with and closest to pure React. An alternative is Next.js, but I think it is too complicated for our use case and requires a bit of time to learn.
• Use vanilla HTML, CSS, and JS with Tailwind and Parcel. This might be one of the easiest ways to start, but it might grow into a big mess.

AI libraries

We can add any AI libs deemed necessary for the project, but to start with, we can:

• Use LangChain - this is a huge collection of high- and low-level tools and building blocks to build apps with LLMs. It has tools to process and embed texts and put them into a vector stores, tools to retrieve data from vector stores, tools to answer questions about a large corpus of text data, to name a few.
• Use Chroma vector store - it is probably the most well-integrated vector store with LangChain and it can run in-memory for quick experiments.
• (optional) LlamaIndex - tools to work with and query documents.

Other tools

TBD

Repository structure

Considering all of this (my preferred choices are emboldened), I would propose two structures:

• Start tiny and go from there (only Streamlit for UI, no FastAPI)

  project_name/
  │
  ├── app/
  │   └── main.py              # main application entry point for Streamlit
  │
  ├── .env                    # environment variables for local development
  ├── .gitignore              # ignored files for Git
  ├── Dockerfile              # Dockerfile to build the container for the application
  ├── docker-compose.yml      # docker-compose configuration file for running services
  ├── requirements.txt        # project dependencies
  └── README.md               # project documentation

• Start with a bit more structure (and use both Streamlit and FastAPI)

  project_name/
  │
  ├── api/
  │   ├── main.py             # main application entry point for FastAPI
  │   ├── models.py           # database (vector store) models / Pydantic schemas
  │   ├── crud.py             # database operations / CRUD functions
  │   ├── api/
  │   │   ├── __init__.py
  │   │   └── endpoint_name.py  # FastAPI routers for specific endpoints
  │   └── utils.py            # utility functions and classes
  │
  ├── frontend/
  │   └── app.py              # main application entry point for Streamlit
  │
  ├── .env                    # environment variables for local development
  ├── .gitignore              # ignored files for Git
  ├── Dockerfile              # Dockerfile to build the container for the application
  ├── docker-compose.yml      # docker-compose configuration file for running services
  ├── requirements.txt        # project dependencies
  └── README.md               # project documentation

Since in most cases we will have separate apps for backend and frontend, we can use Docker and Docker Compose to orchestrate the two. Note that during development we won't need Docker Compose, since the whole dev env will be running in a container.

---

WDYT?

[Konosh93]

@NikitaKurpas, thank you for the excellent comment and detailed information with links. Your input has been very helpful in refining our approach for the project setup. Based on your suggestions, and our subsequent discussions, let me know if you agree with the following updates:

1. For the Editor/IDE, we'll accommodate personal preferences and manage collaboration through Git. We'll decide on the day of the session based on a quick poll.
2. We won't create templates for our projects. Instead, we'll use lightweight frameworks like Streamlit by default and use the first commit of our future projects as a boilerplate for subsequent projects.

We'll keep the suggestions in the issue for future reference as general guidelines. For our first mob programming session, we will create the repository using Streamlit and perform the initial setup on the day of the session.

Thanks again for your valuable input!

P.S. Regarding GH Discussions, I have never used them but they seem to be better for discussions than issues indeed.

[elliexcoding]

Are we going to merge in changes using forks or branches?

[NikitaKurpas]

Are we going to merge in changes using forks or branches?

I was thinking branches, but that means that @Konosh93 will have to add everyone as a contributor. If that is not desirable, then we can use forks instead.