The ability to limit the System Architecture

[SaintDubious]

Requirement - what kind of business use case are you trying to solve?

We'd like to be able to visualize service dependencies for a system with a lot of services (in the 10k range)

Problem - what in Jaeger blocks you from solving the requirement?

The existing System Architecture tab provides a nice view, but there's no way to limit or filter the results. It seems aimed at showing ALL of the services within a system. I haven't actually tried putting all of our services on the page, but I'm fairly certain it wouldn't be useful with that many.

Proposal - what do you suggest to solve the problem or improve the existing situation?

An entry box with an auto-complete (like on the main Search page) that allows me to enter a service of interest. Include this in the URL parameters passed to the data provider (along with the existing "lookback" and "endTS" arguments).

Any open questions to address

Full disclosure: we don't actually use Jaeger as a data provider, we only use jaeger-ui as a front end. As such I'm not terribly fussy about how Jaeger chooses to implement what is shown on the page, I'm mostly interested in a way to let a user select a service, and then provide that selection back to the data layer.

I acknowledge that we're an edge case both in how we use jaeger-ui, and in our size. As such I totally get if you just want to put this on the bottom of the Issue pile. Hopefully no harm in asking, and maybe someone else could find it useful.

Thanks for your consideration.

[jpkrohling]

@esnible, I think we touched on this topic some time ago.

@SaintDubious, do you think the backend should filter the services, or could that be done by the UI itself? Loading a JSON with 10k services shouldn't be a big problem.

[SaintDubious]

@jpkrohling I think the backend would have to do it. The communication layer doesn't just respond with services, it responds with service pairs. Worst case if 10k services all talk to each other that's 10k^2 pairs, which would be an awful lot of data to return.

Just for fun I hooked up a minimal test (by putting a limit on the underlying db call) of a few thousands services. The results look like what I was expecting, a hairball :)

[jpkrohling]

Do you think this would be relevant for the query API, @pavolloffay?

[pavolloffay]

It certainly is a valid requirement and a way to implement it.

[wthrajat]

Hi @SaintDubious I wanted to seek some clarification.

The requirement of being able to visualize a system with a significant number of nodes, such as the 10k range, is clear to me. However, I'm curious about the rationale behind introducing a filtering mechanism when the primary goal is to visualize a substantial number of nodes. It seems that filtering (I'm calling this feature as filtering: an entry box with an auto-complete that allows to enter a service of interest) might ultimately result in displaying a smaller subset of nodes, which raises the question of the intended use case for such an option.

Could you kindly provide more insight into how filtering through services aligns with the objective of visualizing a large number of nodes? What I can guess is that even after filtering, we will still have a very large amount of nodes (10k range maybe).

Thanks a lot. (cc @yurishkuro)

[yurishkuro]

@wthrajat good point, the OP did not really specify what they expect to see after applying the filtering.

I see a couple different ways of implementing a filter:

1. it highlights the service(s) matching the filter and grays out the other services. This is more interactive and does not change the graph or the # of nodes displayed.
2. it removes all other services except those matching the filter and their neighbors up to N hops away (N would be another filtering parameter, default to 1). This could be implemented via server filtering as discussed above (to minimize the volume of data sent to the frontend, although I am skeptical it would be a problem for modern computers - somewhat depends on the naming scheme used for services), but it would be visually very disruptive as the graph would have to change for every filter value.

[wthrajat]

I see. I happen to have the working solutions for both of the approaches right now with me (not inside jaeger-ui though as of now; but let me know if you want to see them right away).

I recently did my GSoC project in the same topic i.e., data visualization so I have an idea. I plan to put the potential solutions in my LFX proposal.

But my doubt is cleared now that we can have either of these 2 approaches. Thanks a lot @yurishkuro :)