Beckn Onix: Network observability dashboard for Beckn - Product Requirements ver 1.1

emmayank commented 2 months ago

Description

Develop a comprehensive Network Observability Server to provide real-time and historical insights into the operations of the Beckn network. This server will address key challenges in visibility, scalability, compliance, auditing, and proactive monitoring. It will collect data from Beckn Adapters using OpenTelemetry, process and store this data, and display it on a user-friendly dashboard.

Relates here: https://github.com/beckn/beckn-utilities/issues/110

Goals

[ ] Implement data collection and ingestion system using OpenTelemetry
[ ] Develop a scalable backend for data processing and storage
[ ] Create a user-friendly dashboard for data visualization
[ ] Ensure data anonymization and aggregation for privacy compliance
[ ] Implement export functionality for data analysis
[ ] Provide documentation for Beckn Adapter integration
[ ] Design and implement a modular and extensible architecture
[ ] Ensure system scalability to handle growing network demands

Expected Outcome

A fully functional Network Observability Server that provides valuable insights into Beckn network operations, enhances trust among participants, improves operational efficiency, and supports informed decision-making. The server will offer real-time monitoring, historical data analysis, and customizable dashboards, all while maintaining data privacy and security.

Acceptance Criteria

[ ] Data is successfully collected from Beckn Adapters using OpenTelemetry
[ ] Dashboard displays key metrics: active network participants, search requests, successful responses, and order metrics
[ ] Users can filter and group data by location and domain
[ ] Data can be exported in CSV format
[ ] System demonstrates horizontal scalability without performance degradation
[ ] All displayed data is anonymized and aggregated
[ ] Documentation for Beckn Adapter configuration is complete and clear
[ ] System complies with relevant data protection and privacy regulations
[ ] Dashboard is responsive and accessible on various devices
[ ] RESTful APIs are available for external system integration
[ ] Containerized deployment using Docker and Kubernetes is successful

Mockups / Wireframes

TBD

Product Name

Beckn

Domain

TBD

Tech Skills Needed

React.js, Node.js, OpenTelemetry, PostgreSQL, Elasticsearch, Docker, Kubernetes, Data visualization libraries (D3.js or Chart.js), Apache Kafka or RabbitMQ, Redis, RESTful API design, TLS/SSL and encryption techniques

Complexity

High

Sub Category

Monitoring and Observability

yesrag2309 commented 2 months ago

@ravi-prakash-v had shared a few milestones, with documentation being the first to be addressed. The team has shared the document with Ravi for feedback and further actions. This is the current status of this card as of August 30, 2024.

Note: The document review is ongoing in a separate email thread. @viraj89, please provide additional information on this ticket

@emmayank @faizmagic

viraj89 commented 2 months ago

On Aug 28 the following message was sent to Ravi for the feedback over email:

Hi Ravi, Venkatesh had already documented the informal requirements on the Confluence page last week: Network Observability - Requirements. We might have missed sharing with you earlier. We also prepared the PRD, available here: Google Doc. Venkatesh and I were back in the office today, which is why we couldn't share it on Monday, August 26, as you requested. Please review and provide your feedback.

yesrag2309 commented 2 months ago

@viraj89 -Need to discuss this card on tomorrows 3pm backlog discussion call.

@emmayank @faizmagic @SyedMubarak008

yesrag2309 commented 2 months ago

@viraj89 -Can you please update the current status on of this card

@emmayank @faizmagic

yesrag2309 commented 2 months ago

@viraj89 - Please find below the current week status and next week plan.

Current week status - Discussions have started, and we are awaiting further inputs from Ravi. Next week plan - Planning to establish clarity, gather details, and set priorities for the next iteration.

@emmayank @faizmagic

yesrag2309 commented 1 month ago

@viraj89 - Please update the card. Do we have all the necessary details to start development in the current iteration?

@faizmagic @vbabuEM @emmayank

Kathirvel-ai commented 1 week ago

Current Status:

Assigned to Ragav. Ragav to work with Viraj and get clarity
Review and Update the PRD

@yesrag2309 @viraj89

faizmagic commented 2 days ago

Current Status:

Ragav to update the PRD with the payload details (mapped to the beckn schema) which needs to be send to the the telemetry service
Plan to complete the same by 5th Nov 2024

Next steps

Once the details are captured, review the same with Ravi once he is back from vacation

CC: @ravi-prakash-v

yesrag2309 commented 2 days ago

Current Status:

PRD updated with the payload details

Next steps Need to review the PRD internal Once @ravi-prakash-v is back from vacation need review the PRD with him.

yesrag2309 commented 1 day ago

Current Status:

PRD updated with the payload details

Next steps

Need to review the PRD internal
Once @ravi-prakash-v is back from vacation need review the PRD with him.

yesrag2309 commented 17 hours ago

Current Status:

PRD updated with the payload details and additional scenario details.
As @faizmagic suggested moved all the updated content to Google Doc.

Next steps

@faizmagic please review the google document - link
Once @ravi-prakash-v is back from vacation need review the PRD with him.

beckn / beckn-onix