[DMP 2024]: Add real-time collaboration to Music Blocks

[walterbender]

Ticket Contents

Description

The Python version of Turtle Blocks has a collaboration mode, where the learner can share stacks of code and, in real time, share graphical output. This project is to add similar functionality to Music Blocks. Following the pattern of Turtle Blocks collaboration is pretty straight-forward. What makes this project difficult is that Music Blocks will require a higher degree of synchronization.

Specifically, we would be working toward accomplishing the following:

Server-side data storage and proxying
API to syncrhonize Music Blocks projects across browsers
Features to perform projects in sync, in real time, with low latency

Goals & Mid-Point Milestone

Goals

• [ ] [Design collaboration architecture]
• [ ] [Build backend service]
• [ ] [Build frontend UX]
• [ ] [Tests/maintenance]
• [ ] [Documentation]
• [ ] [Goals Achieved By Mid-point Milestone]
• [ ] [Design collaboration architecture]
• [ ] [Build backend service]

Setup/Installation

No response

Expected Outcome

• Any user should be able to select/offer a project for collaboration.
• Any user should be able to browse for projects available for collaboration.
• Once collaboration begins, as users join, they start from the current state of the shared project.
• Changes by any user are shared to all users.
• Collisions on merge are resolved by giving precedence based on the order of joining the collaboration -- first to join has priority.

Acceptance Criteria

A working model of collaboration. Scaling is outside of the scope of this initial implementation.

Implementation Details

We'll need to run a server in a docker running on one of the Sugar Labs servers. We can use the existing Planet to store the shared project.

Mockups/Wireframes

No response

Product Name

Music Blocks

Organisation Name

Sugar Labs

Domain

⁠Education

Tech Skills Needed

Docker, JavaScript, Python, WebSockets

Mentor(s)

@pikurasa @walterbender

Category

Backend, Frontend, Research

[rohitkori]

Hi @walterbender, I am interested in this project. I have experience working on various projects and have worked with various tech stacks like Node.js, Django, and ReactJs, as well as DevOps tools such as Docker and Kubernetes. Additionally, I am one of the coordinators of the open-source community at my college and have worked on open-source projects. Could you please provide me with more details about the project description and approach? I would appreciate your help in gaining further insights.

[walterbender]

@rohitkori Please refer to the Getting Started section of our Docs repo.

[AkanshuAich]

Hii @walterbender ,

I am Akanshu Aich, a third year student from International Institute of Information Technology, Bhubaneswar. I am writing to express my interest in contributing to this project as a part of DSP 2024. Having thoroughly reviewed the project, I am impressed by its objectives and it seeks the potential for great impact in industries.

With my background in Backend using Django , MERN with practicing hands on Machine learning and DevOps such as Docker, I believe I can make valuable contributions to both backend and frontend part. My experience includes several projects like Society-Expenditure Manager using Django, Real Estate using MERN and Info-Finding Tool using Machine Learning(LLM), which I believe align well with the goals of your project.

I am particularly interested in fulfilling the requirements of the project and have some ideas on how to approach it effectively. I am committed to adhering to best practices, contributing high-quality code, and actively collaborating with the project maintainers and community.

I am excited about the opportunity to contribute to Music Blocks and help further its mission. I look forward to discussing potential contributions and how I can best support the project.

Please guide me with procedure and with all your knowledge and experience.

[walterbender]

@AkanshuAich I would recommend that start off by creating some Music Blocks projects to get a feel for how the language works. And explore some of the example projects. Then take a look at the Getting Started guidelines in the Docs section of github.com/sugarlabs.

[apsinghdev]

@walterbender can we start making PR's for this project?

[walterbender]

PRs are always welcome.

[ashuashutosh2211]

@walterbender I'm Ashutosh, a prefinal year student at IIT Jodhpur, specializing in Artificial Intelligence and Data Science. I have a strong proficiency in programming languages such as Python and C++. My experience includes working on diverse projects spanning machine learning and deep learning, including endeavors like Stock Price Prediction and Speech-to-Text Transcription. In addition to my programming skills, I have hands-on experience with various databases such as SQL (MySQL), Document-Oriented Databases (MongoDB), and Graph Databases (Neo4j). One notable project where I applied these skills is the development of a Video Search Engine. I am interested to contribute in this project. Could you please provide me with more details about the project description and approach? I would appreciate your help in gaining further insights.

[walterbender]

@ashuashutosh2211 I recommend that you (1) familiarize yourself with Music Blocks -- write some Music Blocks code; (2) think about how multiple people could collaborate; (3) think about collaboration frameworks.

[NEGRAH-S]

Hello @walterbender , Am Highly interested for this project and will send you my apporaoch for same along with a draft proposal for reviews by tonight.

[RSN601KRI]

Hello, @walterbender , I would like to contribute to this project. I have the required skillset for it. I'm glad to work and explore as well as express through my ideas and skillset.😊

[walterbender]

@NEGRAH-S Looking forward to hearing from you and reviewing your ideas.

[walterbender]

@RSN601KRI The project will involve some experience with JavaScript for the front end. Probably the backend would require Python skills. There would need to be some server component through which the users communicate.

[RSN601KRI]

Sure @walterbender, the project mainly focuses on using JavaScript for the front end to make a cool and easy-to-use interface. We'll also need Python skills for the backend stuff, like handling data and making sure everything runs smoothly behind the scenes. Plus, we'll set up a server so users can talk to each other. To sum up, being good at both JavaScript and Python is key. I've got a proposal ready. Could you give it a look so I can get started on the project?

[walterbender]

@RSN601KRI feel free to send me your proposal. (walter --AT-- sugarlabs --DOT-- org)

[AbhimanyuSamagra]

Do not ask process related questions about how to apply and who to contact in the above ticket. The only questions allowed are about technical aspects of the project itself. If you want help with the process, you can refer instructions listed on Unstop and any further queries can be taken up on our Discord channel titled DMP queries. Here's a Video Tutorial on how to submit a proposal for a project.

[walterbender]

Weekly Goals (To be filled by mentor)

Week 1

• [x]  Sync up goals with MB team
• [x] Add a collaboration starter button

Week 2

• [x]  Create a minimal frontend for the collaboration room
• [x] just a blank screen with exit button that pops up on clicking the collaborate button
• [x] Setup the local server
• [x] Install and configure Socket.io and Yjs to server side
• [x] Implement server logic

Week 3

• [x] Install Yjs and Socket.io libraries
• [x] Configure the libraries to client side

Week 4

• [x] Initialize socket connection
• [x] Implement rooms and broadcast logic
• [x] Create a room automatically on collaboration initialization
• [x] User joins in the room who hit the room_id endpoint
• [x] User gets the broadcasted changes in the room.
• [x] Implement logic to Send/Receive changes in the document
• [x] Create a Yjs doc using the Project's data
• [x] Setup the Doc to be tracked
• [x] Track the changes and send over the server to the particular room.

Week 5

• [x] Define changes to track and send/receive.
• [x] adding a block (to the interface)
• [x] connecting a block (in the stack)
• [x] removing a block (from the interface)
• [x] disconnecting a block (from a stack)
• [x] change the value of a block
• [x] Implement Logic to generate a block to all the connected clients if any one of the client has created that block.

Week 6

• [x] Implement logic to send the changes of removing a block
• [x] Implement logic to send the changes of connecting a block in stack
• [x] Implement logic to send the changes of disconnecting a block from the stack
• [x] Implement logic to send the changes of updating value of a block
• [x] Implement logic to send the changes of moving a block from one position to another

Week 7

• [x] Analyse collaboration design in Turtle Blocks
• [x] Design collaboration flow for Music Blocks
• [x] Decided the link Mechanism for near future

Week 8

• [x] Implement Link generation Mechanism
• [x] Create an endpoint to server side to open project in collaboration
• [x] Implement room logic to make a user join in a specific room
• [x] Implement server side logic to send changes in that room

Week 9

• [x] Fix multiple mouse bug
• [x] Implement mechanism to show link on clicking collaborate
• [x] Implement logic to redirect the user to the collaboration project
• [x] Start that project having collaboration on in a particular room

Week 10

• [x] setup and explore planet-server locally
• [x] Add logic to delete a room if there is no user in it
• [x] Implement a collaboration mouse
• [x] Create a container that stores all the rooms with the users they contain (to the server side)
• [x] Create a set container that stores all the socketIDs (users) to track their movement
• [x] Implement the logic to create existing-cursor to the new joiner
• [x] Implement the logic to create a new-cursor to all the connected users
• [x] Add logic to remove the respective cursor from all the client's screens of the user who got disconnected.

Week 11

• [x] Complete the implementation of the collaboration mouse
• [x] Implement creating a new cursor representing the new joiner
• [x] Implement tracking and sending the position value of a mouse to all the connected clients
• [x] Implement removing the corresponding mouse when a user disconnects.
• [x] Set up a container that stores the names of connected clients (client side)
• [x] Implement storing a room along with socketID in a global rooms container (server-side)
• [x] Add the logic to store the names (server-side)
• [x] Add logic to clear the data (server and client side) when a user disconnects.
• [x] Plan the dockerization and deployment of the collaboration server

Week 12

• [x] Dockerize the collaboration-server
• [x] Implement server side logging mechanism
• [x] Change the CORS headers for security
• [x] Make the port values configurable via environment variable
• [x] Cleanup the unused code/libs/etc.
• [x] Remove the testing Collaboration Button

[apsinghdev]

Weekly Learnings & Updates (To be filled by contributor)

Week 1

• Learnt the difference between socket.io library and WebSockets
• Learnt SOLID principles to apply in further implementation.

  Week 2

• Leant installing and configuring the socket.io and Y.js libraries.
• Learnt how to implement basic server-side logic using socket.io and Y.js libraries.
• I learned how to interpret and understand nested method calls in object-oriented programming, where methods are accessed through a hierarchy of objects. i.e. Planet.GlobalPlanet.ProjectViewer.open(this.id);

  Week 3

• Learned about minified and unminified versions of libraries and the differences between them.
• Learned to minify a library using Gulp-Uglify.
• Understood the Implementation of rooms and namespaces using Socket.io.
• Gained an understanding of CommonJS and ES module systems.
• Studied the UMD pattern for libraries.
• Learned to bundle libraries using Webpack and Rollup.
• Installed and integrated UMD libraries into a project.

  Week 4

• Learned creating Yjs documents
• Gained an understanding of creating a socket server
• Understood how to connect multiple clients to the socket server
• Learned converting a JS object into a Yjs Doc

  Week 5

• Studied implementing call limiting mechanism to avoid unnecessary calls if socket server not found
• Parsing data into JSON and vice versa
• Emitting socket event when a particular change happens
• Broadcasting JSON data over the socket server to the connected peers
• Receiving and applying the JSON data to the connected clients

  Week 6

• Learned how to convert the received JSON data into Music Block's blocks.
• Read about how to combine multiple emit events into one and implement it.
• Learned the use of getters and setters in javascript.
• Understood the nature of the asynchronous programming.

  Week 7

• Learned about the collaboration in Turtle Blocks
• Read docs of Sugar
• Understood the architecture of the collaboration in TB
• Learned about rooms in socket.io

  Week 8

• Understood the concept of namespaces in socket.io
• Read about creating a unique ID creation mechanism using the current timestamp
• Learned about the working of socket.io in a react app

  Week 9

• Learned about the async nature of javascript in-depth
• Understood the concept of query parameters in a URL and implemented in the collaboration link
• Learned about caching and client-side storage
• Read about localStorage, sessionStorage, and indexedDB API

  Week 10

• Learned to use MySQL DBMS by creating databases importing data and migrating schema
• Explored the Planer-Server of Music Blocks
• Learned to measure the resource (RAM, disk, GPU) requirements for the project's server
• Read about server-side event emitting and implemented

  Week 11

• Understood measuring RAM usage per request to the server
• Learned about sunjammer, server of sugar labs
• Read about use cases of nginx and Apache servers
• Got to know about the difference between static content/websites and dynamic content/websites
• Explored the difference between proxy and reverse proxy
• Learned about bandwidth and its use cases (ie. video streaming platforms).
• Learned the use of map and set data structures in javascript

  Week 12

• Read about throttling and debouncing and applied to throttle for in collaboration-server.
• Learned about position value relative to viewport and account of scrolling in it.
• Read about server-side logging and Configured logging using morgan for collaboration-server
• Learned to make docker images implemented it and pushed the images to the dockerHub.

[apsinghdev]

Link of the collaboration-server - https://github.com/sugarlabs/collaboration-server