Implementation of network observability dashboard for Beckn

[emmayank]

Description

Develop a comprehensive Network Observability Server to provide real-time and historical insights into the operations of the Beckn network. This server will address key challenges in visibility, scalability, compliance, auditing, and proactive monitoring. It will collect data from Beckn Adapters using OpenTelemetry, process and store this data, and display it on a user-friendly dashboard.

Goals

• [ ] Implement data collection and ingestion system using OpenTelemetry
• [ ] Develop a scalable backend for data processing and storage
• [ ] Create a user-friendly dashboard for data visualization
• [ ] Ensure data anonymization and aggregation for privacy compliance
• [ ] Implement export functionality for data analysis
• [ ] Provide documentation for Beckn Adapter integration
• [ ] Design and implement a modular and extensible architecture
• [ ] Ensure system scalability to handle growing network demands

Expected Outcome

A fully functional Network Observability Server that provides valuable insights into Beckn network operations, enhances trust among participants, improves operational efficiency, and supports informed decision-making. The server will offer real-time monitoring, historical data analysis, and customizable dashboards, all while maintaining data privacy and security.

Acceptance Criteria

• [ ] Data is successfully collected from Beckn Adapters using OpenTelemetry
• [ ] Dashboard displays key metrics: active network participants, search requests, successful responses, and order metrics
• [ ] Users can filter and group data by location and domain
• [ ] Data can be exported in CSV format
• [ ] System demonstrates horizontal scalability without performance degradation
• [ ] All displayed data is anonymized and aggregated
• [ ] Documentation for Beckn Adapter configuration is complete and clear
• [ ] System complies with relevant data protection and privacy regulations
• [ ] Dashboard is responsive and accessible on various devices
• [ ] RESTful APIs are available for external system integration
• [ ] Containerized deployment using Docker and Kubernetes is successful

Mockups / Wireframes

TBD

Product Name

Beckn

Domain

TBD

Tech Skills Needed

• React.js, Node.js, OpenTelemetry, PostgreSQL, Elasticsearch, Docker, Kubernetes, Data visualization libraries (D3.js or Chart.js), Apache Kafka or RabbitMQ, Redis, RESTful API design, TLS/SSL and encryption techniques

Complexity

High

Category

Infrastructure

Sub Category

Monitoring and Observability

[yesrag2309]

@ravi-prakash-v had shared a few milestones, with documentation being the first to be addressed. The team has shared the document with Ravi for feedback and further actions. This is the current status of this card as of August 30, 2024.

Note: The document review is ongoing in a separate email thread. @viraj89, please provide additional information on this ticket

@emmayank @faizmagic

[viraj89]

On Aug 28 the following message was sent to Ravi for the feedback over email:

Hi Ravi, Venkatesh had already documented the informal requirements on the Confluence page last week: Network Observability - Requirements. We might have missed sharing with you earlier. We also prepared the PRD, available here: Google Doc. Venkatesh and I were back in the office today, which is why we couldn't share it on Monday, August 26, as you requested. Please review and provide your feedback.

[yesrag2309]

@viraj89 -Need to discuss this card on tomorrows 3pm backlog discussion call.

@emmayank @faizmagic @SyedMubarak008

[yesrag2309]

@viraj89 -Can you please update the current status on of this card

@emmayank @faizmagic

[yesrag2309]

@viraj89 - Please find below the current week status and next week plan.

Current week status - Discussions have started, and we are awaiting further inputs from Ravi. Next week plan - Planning to establish clarity, gather details, and set priorities for the next iteration.

@emmayank @faizmagic