Provide top level view of the application

[gireeshpunathil]

I have had discussion with @cvignola on this. This is a tracker issue for multiple work items under this umbrella.

Problem: Microservice based logging do not provide top level view of the application. While microservice based architectural pattern has improved many aspects of the software development, deployment and operations, the need to reconstruct the bigger picture from bits and pieces of broken up diagnostic data is an unforeseen side effect. Traditional monoliths never had this issue.

Solution: Aggregate the content from across microservices and display it in common filterable dashboards.

This breaks up into multiple independent work items:

• Ability to label deployments and services with a name that identifies the top level application that this service or deployment belongs to.
• Ability for the kappnav operator to collect, process and send aggregate data
• Ability for the kappnav operator to collect, process and send cluster life cycle data
• Ability for the UI to render aggregate data in common filterable dashboards
• A data processing layer that define meaningful aggregations. Example includes:
  • All data sequenced against time
  • Metadata derived from analyzing logs that provide insights into the state of services
  • Application data integrated with cluster life cycle data

Would like to hear from this community to ratify the intent, before start working on this!

[gireeshpunathil]

please assign this me!

[cvignola]

@gireeshpunathil Thank you for your interest in the kAppNav project and thank you for sharing your ideas: I like the goals of this issue. However, what you're describing sounds a lot like what can be accomplished using other open source technologies like Prometheus, Grafana, and possibly Jaeger.

So rather than build capabilities into kAppNav similar to what is possible with open source technologies like the ones mentioned previously, perhaps a more effective way to achieve the outlined goals of this issue would be to experiment with integrations of kAppNav with those technologies to determine what can be accomplished through them.

The integration with those other technologies might result in new instructions, new install scripts, new kAppNav custom actions, and maybe even new kAppNav customization hooks in order to achieve the desired level of integration between kAppNav and these other technologies. These integrations assets could be captured and stored as part of the kAppNav project.

Please share your thoughts.

[gireeshpunathil]

@cvignola - thank you! Agree that re-using / integrating with established technologies as much as possible is the right thing to do. However, while assessing these technologies in the context of kappnav, it is possible that newer observable possibilities arise.

Sometime this week, I will open child work items that track experimentation / integration of the said individual technologies with kappnav.

[gireeshpunathil]

I was side-tracked, but going to resume the work on this.