stdlib-js / google-summer-of-code

Google Summer of Code resources.
https://github.com/stdlib-js/stdlib
26 stars 7 forks source link

[RFC]: build a developer dashboard for tracking repository build status #66

Closed AhmedKhaled590 closed 7 months ago

AhmedKhaled590 commented 8 months ago

Full name

Ahmed Khaled Mahmoud

University status

No

University name

Graduated from Cairo University

University program

Computer Engineering

Expected graduation

JUL 2023

Short biography

I am a Software Engineer based in Egypt. I hold a Bachelor's degree in Computer Engineering from Cairo University, where I graduated with a GPA of 3.3 out of 4, receiving recognition for my excellent performance in the graduation project.

Throughout my career, I have gained extensive experience in various technical areas. I have worked as a Netsuite Technical Consultant at eDigits Consulting, where I progressed to the role of Senior Netsuite Technical Consultant. During this time, I led multiple projects and developed expertise in JavaScript, HTML, CSS, Node.js, Git, SuiteScript 2.0 and SuiteApps

Some of my notable projects include my graduation project DesignOder, a web-tool aimed at automating the conversion of wireframes into fully web pages using cutting-edge technologies such as computer vision, natural language processing, and machine learning. Additionally, I contributed to the development of CMPLR Website (one of the projects I worked on during faculty time), a ReactJS-based clone of a Tumblr website.

In my part-time role at Atom BI, I worked as a Junior Software Engineer, focusing on developing a business intelligence tool using GO language, JS, and PostgreSQL.

I have completed several certifications covering web development, backend development and databases to continually enhance my skills and stay updated with industry trends.

Timezone

Egypt (GMT +2).

Contact details

email: kahmd1444@gmail.com, github:AhmedKhaled590

Platform

Linux

Editor

My preferred code editor is Visual Studio Code (VSCode). There are several reasons why I find VSCode to be the best fit for my workflow some of them that it offers excellent support for a wide range of programming languages and frameworks out of the box. Additionally, its vibrant ecosystem allows for easy integration of extensions tailored to specific needs.

Programming experience

Throughout my programming journey, I've had the opportunity to work on various projects that have allowed me to apply my skills and creativity. One notable project I've created is a web-tool called DesignOder.

DesignOder is a tool aimed at automating the conversion of wireframes into fully functioning web pages. The idea behind this project stemmed from the need to streamline the web development process and reduce manual effort in translating design concepts into code.

Here's an overview of the key features and components of DesignOder:

Computer Vision Modules: DesignOder incorporates computer vision techniques to analyze and interpret wireframe designs. This involves detecting various elements such as inputs, containers, images, and checkboxes within the wireframes.

Natural Language Processing (NLP) Module: To enhance user experience, DesignOder utilizes NLP algorithms to understand and interpret textual descriptions associated with different design elements. This helps in generating accurate HTML code based on the wireframe annotations.

Machine Learning Integration: Machine learning algorithms are employed to optimize the conversion process by learning from user interactions and feedback. This adaptive approach enables DesignOder to continuously improve its accuracy and efficiency over time.

Frontend and Backend Integration: DesignOder consists of both frontend and backend components. The frontend interface provides users with an intuitive platform for uploading wireframes, specifying design preferences, and reviewing the generated code. On the backend, sophisticated algorithms process the input data, perform the necessary transformations, and output the corresponding HTML markup.

JavaScript experience

JavaScript experience In my experience with JavaScript, a significant portion of my work has been dedicated to developing solutions within the Oracle Netsuite environment using SuiteScript. SuiteScript is a JavaScript-based scripting language specifically designed for extending and customizing Netsuite, allowing developers to automate business processes, create customizations, and integrate with external systems.

One of the standout features of JavaScript that I particularly appreciate is its versatility. JavaScript is a multi-paradigm language, allowing me to write code using different programming styles, including procedural, functional, and object-oriented programming. This flexibility enables me to approach problem-solving in diverse ways, adapting to the specific requirements of each project.

Moreover, JavaScript's asynchronous programming model, facilitated by features like Promises and async/await syntax, is another aspect that I find immensely valuable. Asynchronous programming allows for non-blocking I/O operations, enabling the creation of responsive and efficient applications that can handle concurrent tasks without blocking the execution flow.

However, if I were to identify a least favorite feature of JavaScript, it would likely be its type coercion and loose equality comparison. JavaScript's automatic type conversion can sometimes lead to unexpected behaviors and errors, especially for developers coming from strongly typed languages. Additionally, the concept of loose equality comparison (==) can result in subtle bugs and inconsistencies, as it performs type coercion when comparing values, often leading to unintended outcomes.

Despite these challenges, JavaScript remains a powerful and widely-used language in the world of web development, offering an extensive ecosystem of libraries, frameworks, and tools that continue to evolve and enhance its capabilities. Overall, my experience with JavaScript has been positive, and I continue to appreciate its strengths while navigating its nuances and challenges.

Node.js experience

Node.js experience During my academic tenure, I worked on projects that provided hands-on experience in backend development using Node.js. One such project was the Blood Bank Website.

For the Blood Bank Website project, I was responsible for designing both the frontend and backend components. On the frontend, I utilized HTML, CSS, JavaScript, and Bootstrap 4 to create a visually appealing and user-friendly interface. This involved structuring the layout, styling elements, and incorporating interactive features to enhance user engagement.

On the backend, I employed Node.js as the server-side runtime environment to handle server logic and data management. Node.js enabled me to develop backend functionalities, such as user authentication, data validation, and API endpoints. Additionally, I utilized SQLite as the database management system to store and manage data related to blood donors, recipients, and inventory.

C/Fortran experience

N/A

Interest in stdlib

Although I haven't had the chance to utilize stdlib in my work, upon reviewing its capabilities briefly, I am impressed by its comprehensive nature, which encompasses a broad spectrum of functionalities.

Version control

Yes

Contributions to stdlib

Pull Request #1369: Added support for forEachRight method (RFC #5678) - Closed

This PR proposed adding support for invoking a callback for each visual character of a string while iterating from right-to-left. It has been closed.

Goals

STDLIB

[RFC]: build a developer dashboard for tracking repository build status

Introduction

This proposal aims to enhance the integrity and productivity of the Standard Library (stdlib) project by developing a comprehensive Developer Dashboard for real-time monitoring of build failures across its vast ecosystem of over 3500 repositories. Leveraging modern technologies and best practices, the project seeks to provide stdlib developers with a user-friendly interface to swiftly detect, analyze, and resolve build failures, thereby minimizing downstream disruptions and fostering a collaborative development environment. The proposed solution entails the creation of a Node.js backend for querying build results from a PostgreSQL database, coupled with a responsive frontend dashboard for visualizing and interacting with the data. Through iterative development, rigorous testing, and continuous feedback integration, the Developer Dashboard aims to significantly improve visibility, responsiveness, and efficiency within the stdlib community, ultimately enhancing the overall stability and reliability of the project.

Objectives

The main aim is to establish a web-based dashboard showcasing real-time build failure data for independent repositories within stdlib. The project will be segmented into smaller tasks to streamline the process, ensuring each pull request is self-contained, well-documented, and thoroughly tested. These tasks include:

  1. Initializing environment setup and database connections .
  2. Creating backend APIs to facilitate build result queries .
  3. Designing the frontend layout and initial interface .
  4. Implementing filtering features and repository navigation .
  5. Incorporating historical overviews and supplementary metrics .
  6. Conducting comprehensive testing, gathering feedback, and making refinements .

Technical Approach

  1. Initializing environment setup and database connections

    • Using Makefiles in this way provides a standardized and easily reproducible way to manage environment setup tasks. Developers can simply run make setup to ensure they have all the necessary dependencies installed and configured in a consistent manner. Additionally, Makefiles can be version controlled alongside your project, ensuring that setup instructions are always up-to-date and accessible to all team members.

    • Below is a pseudo example for how the makefile will look like for example

      code

    • Establishing Database Connections

      Option 1: Sequelize ORM:

      • Sequelize is a popular Node.js ORM that supports multiple SQL databases, including PostgreSQL and SQLite.
      • Configure Sequelize to establish connections to the PostgreSQL database using connection parameters such as host, port, username, password, and database name.
      • Define Sequelize models to represent database tables and define associations between them.
      • Utilize Sequelize's migration feature to manage database schema changes and ensure consistency across environments.

      Option 2: Native PostgreSQL Driver:

      • Alternatively, you can use the native PostgreSQL driver for Node.js to establish direct connections to the database.
      • Install the pg package, which provides a simple interface for interacting with PostgreSQL databases.
      • Manually create database connection pools and manage connections within your Node.js application.
      • While this approach offers more control, it may require more low-level configuration compared to using Sequelize ORM.
  2. Creating backend APIs to facilitate build result queries

    1. RESTful API Design:

      • Design a set of RESTful APIs with clear URL structures and HTTP methods for querying build results.
      • Define endpoints for querying build status, retrieving build logs, fetching build artifacts, and filtering build results.
    2. Pagination:

      • Implement pagination for endpoints returning large result sets to improve performance and reduce response times.
      • Allow clients to specify parameters such as page size and page number to control the amount of data returned.
      • Use cursor-based pagination for efficient navigation through large datasets, avoiding issues with offsets and limits such as Performance Degradation, Inconsistent Results.
    3. Testing with Jest:

      • Write comprehensive unit tests using Jest to validate the functionality of each API endpoint.
      • Test for both positive and negative scenarios, including edge cases and error conditions.
      • pseudo example for testing file

        code 1

    4. Documentation:

      • Document the backend APIs thoroughly, including descriptions of each endpoint, expected request and response formats, and sample usage scenarios.
      • Automate API documentation generation using tools like Swagger or OpenAPI to ensure consistency and keep documentation up-to-date.

        code 2

  3. Designing the frontend layout and initial interface

    1. Header Navigation

      1. Provide navigation links to essential sections of the dashboard such as Home, Builds, and Repositories.
      2. Consider using icons or icons alongside text for improved usability and aesthetics
      3. Implement a notifications icon or bell icon in the header to alert users of important updates or notifications.
    2. Home Page Design:

      1. Repository Overview Section:
        • Implement a grid or card layout to display all repositories with their related summary data, such as build status, last commit date, and code coverage percentage.
      2. Summary Data Display:
        • Fetch repository data from the backend API using asynchronous requests with Axios or Fetch API.
      3. Search and Filtering Functionality:
        • Integrate a search bar at the top of the page to allow users to search for specific repositories by name or keywords.
        • Implement filtering options, such as by build status (success, failure, in progress) or code coverage percentage.
      4. Pagination:
        • Implement pagination to manage large datasets of repositories efficiently.
        • Use a paginated API endpoint on the backend to fetch repositories in chunks, and implement client-side pagination controls to navigate through pages.
      5. Interactive Elements:
        • Include interactive elements within repository cards, such as clickable links to navigate to the repository on GitHub or view detailed metrics.
        • Utilize hover effects or tooltips to provide additional context or information about each repository.
        • This is a quick hand-on wireframe for the very high level layout

      image

    3. Detailed Repository Page:

      1. Overview Section:
        • Display essential information about the repository, such as name, description, owner, and URL.
        • Include badges or icons to represent the current status of the repository (e.g., build status, code coverage).
      2. Commit History:
        • Render a list or timeline of recent commits to the repository, including details such as commit message, author, timestamp, and associated branch or tag.
        • Provide options to filter commits based on criteria such as author, branch, or date range.
        • Allow users to click on individual commits to view more detailed information or navigate to related resources.
        • Add button to view whole list of commits related to this repository in a separate page, this page will have detailed data for each commit, filtering options based on specific date or user for example, this approach will be applied in all upcoming views below
      3. Build Status:
        • Show the current status of the repository's builds, indicating whether they are successful, failed, or in progress.
        • Include information about the latest build, such as build number, duration, and triggered events.
        • Provide links to view build logs or artifacts for further analysis.
      4. Event Tracking:
        • Display a list of recent events related to the repository, such as pull requests, issues, or comments.
        • Include details about each event, such as event type, timestamp, user, and associated action or description.
        • Allow users to filter events based on event type or timestamp and search for specific events using keywords.
      5. Workflow Run and Job Details:
        • Show information about recent workflow runs and associated jobs for the repository.
        • Include details such as workflow name, status, start time, end time, and triggered events.
        • Provide links to view detailed logs, artifacts, or job statuses for individual workflow runs and jobs.
      6. Code Coverage Report:
        • Display code coverage metrics for the repository, including statements, lines, branches, and functions coverage.
        • Visualize code coverage data using charts or graphs to provide insights into code quality and test coverage.
        • Include button to view historical code coverage trends over time.
      7. Navigation and Interaction:
        • Include navigation links or tabs to easily switch between different sections of the repository page.
        • Provide options to customize the layout or view settings based on user preferences to show or hide some sections of the page.
      8. NPM Publish and Download Metrics
        1. Publish Metrics:
          1. Display statistics related to NPM package publishing, such as total number of publishes, latest version, and publishing frequency.
          2. Show details about the most recent publish, including version number, Node.js version compatibility, license, and tarball size.
          3. Provide insights into publishing trends over time, such as monthly or yearly publishing frequency.
        2. Download Metrics:
          1. Present statistics on NPM package downloads, including total download count, daily or weekly download trends, and download distribution across versions.
          2. Show information about the most downloaded package versions, including version number, download count, and download rate.
        3. Export and Sharing Options:
          • Enable users to export publish and download metrics data in various formats (e.g., CSV, JSON) for further analysis or reporting.

Why this project?

This project excites me because it addresses a real need within the stdlib ecosystem: quickly detecting and resolving build failures. As a developer, I know the frustration of dealing with build issues, and having a real-time dashboard to monitor and tackle them head-on is invaluable. Plus, the opportunity to work with cutting-edge frontend technologies like ESBuild and tailwind adds an extra layer of excitement. Overall, it's a project that combines practical impact with the chance to explore new tools and techniques, making it both fulfilling and intellectually stimulating.

Qualifications

With a diverse range of experiences spanning from software engineering to project management, I bring a unique blend of technical expertise and leadership skills to the table. My background includes:

Full Stack Software Engineer: During my time at Atom BI, I worked on developing a business intelligence tool using GO, JavaScript, and PostgreSQL. This experience equipped me with hands-on skills in building scalable and efficient software solutions.

Web Development Experience: As a member of the Web Development Team at Google DSC, I contributed to projects involving course enrollment systems and quiz creation pages. This experience demonstrates my proficiency in frontend development using technologies like ReactJS.

Soft Skills: Throughout my career, I have been involved in managing projects, from designing custom reports to creating integrations between systems. This experience has sharpened my ability to handle diverse challenges and deliver solutions on time and within budget.

Technical Skills: My skill set encompasses a wide range of technologies including JavaScript, HTML, CSS, Node.js, Git, and SQL, among others. I am also familiar with frameworks like React and have experience with databases like PostgreSQL.

Educational Background: I hold a degree in Computer Engineering from Cairo University, where I graduated with a GPA of 3.3 out of 4. My coursework included a diverse range of subjects, from programming languages to algorithms and data structures.

Faculty Projects: During my time at faculty I worked on a some frontend and backend projects using ReactJS and NodeJS, some of them are CMPLR, Reserva and Blood Bank Project

Overall, my combination of technical proficiency, project management experience, and educational background makes me well-suited to tackle the challenges presented by this project. I am excited about the opportunity to leverage my skills and contribute to the development of a real-time developer dashboard for stdlib.

Prior art

npm Status Board

The npm Status Board project serves as an excellent example of a developer dashboard aimed at providing real-time insights into the status of npm services. This project, developed by npm, Inc., offers a comprehensive dashboard that monitors the availability and performance of npm's infrastructure, including registry, website, and authentication services.

Commitment

I'm a full-time software engineer at eDigits Consulting, but I am fully committed to dedicating approximately 20-25 hours per week to the project during the Google Summer of Code program. While managing a full-time job alongside the GSoC program may seem challenging, I am confident in my ability to effectively balance my responsibilities and allocate sufficient time to make meaningful contributions to the project. During the proposal creation period, I was able to efficiently manage my time and complete the proposal while fulfilling my obligations at work. This experience serves as a testament to my ability to prioritize tasks, manage deadlines, and effectively allocate time for both work and personal projects. Besides that I don't plan for any vacations during this period. Overall, I am dedicated to the success of the project and am committed to investing the necessary time and effort to make meaningful contributions during the GSoC program

Schedule

Community Bonding Period and Prepwork Task: During the community bonding period, I plan to familiarize myself with the technologies and tools outlined in the proposal, such as Sequelize ORM and Jest. I will also review relevant database design and actual data.

Week 1-2: Initialization and Backend Setup Task: Set up development environment, including installing necessary dependencies and configuring database connections. Deliverables: Completed environment setup with Makefile for standardized setup tasks. Established connections to PostgreSQL database using Sequelize ORM or native PostgreSQL driver. Estimated Time: 15-20 hours

Week 3-4: Backend API Development Task: Design and implement RESTful APIs for querying build results, including endpoints for build status, logs, and artifacts. Deliverables: Implemented backend APIs with clear URL structures and HTTP methods. Pagination feature for efficient handling of large datasets. Estimated Time: 20-25 hours

Week 5-6: Frontend Layout and Initial Interface Task: Design the layout and user interface for the frontend dashboard, focusing on the home page and repository overview section. Deliverables: Designed responsive layout with navigation links and repository cards. Implemented search and filtering functionality for repositories. Estimated Time: 20-25 hours

Week 7-8: Detailed Repository Page Task: Develop detailed views for individual repositories, including commit history, build status, event tracking, and workflow run details. Deliverables: Completed repository page with all necessary sections and functionalities. Integrated interactive elements for navigation and customization. Estimated Time: 25-30 hours

Week 9-10: Code Coverage and NPM Metrics Task: Implement code coverage report and NPM publish/download metrics sections for the repository page. Deliverables: Added code coverage visualization and historical trends feature. Integrated NPM publish/download metrics with export and sharing options. Estimated Time: 25-30 hours

Week 11-12: Testing, Documentation, and Refinement Task: Conduct comprehensive testing, document backend APIs and frontend components, and make refinements based on feedback. Deliverables: Completed unit tests using Jest for backend APIs. Thoroughly documented APIs and frontend components. Addressed any feedback or issues identified during testing. Estimated Time: 20-25 hours

Related issues

No response

Checklist

Planeshifter commented 8 months ago

You have a strong background and give specific examples of previous projects and technical skills that align with the project's requirements, which is good. However, given that GSoC has historically been reserved for students and is still meant for newcomers to open-source, it would be good to reflect on that in your application and make this case for yourself, showing that you fit the admission criteria.

As for the proposal, it would be strengthened by showing that you have done your homework and resarched the stdlib way of doing things: For example, we would likely want to use tape as a test runner for the backend given that it is what we use in the main project. For testing React components in the front-end code, a reasonable case can be made for Jest as tape does not work too well for this usage, but this case would have to be made explicitly.

It would also be good to add a section on the risks and how you would handle falling behind your proposed schedule, especially in light of your full-time work committment.

kgryte commented 8 months ago

@AhmedKhaled590 You also proposed https://github.com/stdlib-js/google-summer-of-code/issues/68; would you mind specifying which of the two proposals is your preferred choice?

AhmedKhaled590 commented 8 months ago

@Planeshifter @kgryte Thank you for your comments, my preferred proposal choice is #68

kgryte commented 8 months ago

You are free to submit multiple proposals on the GSoC website, but, in each, you should state your preferred project.

AhmedKhaled590 commented 8 months ago

That's a nice info, thank you for sharing.