Visualization of MCParticle + RecoParticle

[brauliorivas]

BEGINRELEASENOTES

• Use a visualization algorithm for graphs.
• Render MCParticle and ReconstructedParticle including the MCRecoParticleAssociation.
• Fix loading particles (I was not taking into account that some associations are across different collections, even collection ids, but now I've considered this aspect)

ENDRELEASENOTES

I've started working on visualization for ReconstructedParticle and the MCRecoParticleAssociation, but more importantly, how to place around these objects. Hence, it's easy to see relations between the same types of objects and across them. Even though we first agreed on placing both objects on a column (MC and Reco), I think we should think of a way to place these objects no matter what type (in the future, when more collections get added, we have to make sure that its well suited for any kind of collections). So after some research, I found out that there exist some different types of algorithms that could help us to make these visualizations. I've read this paper about force-directed graphs, which seem to be useful for EDM4hep, because works well for showing relations of general graphs and depending of which type, may also cluster them (which is similar to EDM4hep). I'll continue learning more about this and implement the one that we like the most (forceatlas2 is a good candidate).

[tmadlener]

I have only briefly skimmed the paper you have linked and I probably have missed quite a few (i.e. almost all) of the details. One question in this context is the following: The paper talks about undirected graphs, but most (I think all of) our graphs actually are directed, since the relations have a clear direction. Am I missing something obvious here, why we would treat them as undirected graphs?

Specifically for this visualization, what I would do is the following

• Draw the MC graph
• Draw the ReconstructedParticle graph (from a user selected collection name)
• Use the MCRecoParticleAssociation collection (again user specified name) to draw links between the elements of the two graphs

This should give us a version 0 of how this would look like to iterate on. It's probably quite horrible, but it serves as a basis :)

Immediate additional functionality that I can think of:

• Prefilter the MC graph, such that only stable (i.e. detectable) particles remain (we have to figure out which generator status this corresponds to)
• Add a toggle to "gray out" MCParticles and reco particles that have no counterpart in the other graph (maybe make this a selector where we can choose between: fully drawn, grayed out, fully removed)
• Add the possibility to load a second MCRecoParticleAssociation collection to draw more connections (because it might be that the connections depend on whether you start on the MC side or on the reco side, even though the association does not necessarily have a direction). This one I would keep at low priority for the actual implementation, but make sure it's not impossible from a design point of view.

Having written down all that, it might be easiest to first make another PR (contrary to what I said during the meeting on Friday), where we only draw the reconstructed particles. To be most useful, this would probably require two main features. Drawing of either one or multiple reconstructed particle collections; Here one would be for PandoraPFOs or ReconstructedParticles as likely default names, but we can make the names user input. Multiple collections would most likely be the former, plus an additional Jets collection. Here the objects in the first collection are probably just a list, whereas, the objects of the Jets, will have so called constituents (stored in the particles field), that are objects of the first collection. The second nice feature would then be, that if I click on one jet object, all the things that are not related to that are grayed out or hidden from the view entirely, but that can also go into another PR, once the basic displaying works.

[github-actions[bot]]

PR Preview Action v1.4.7 :---: Preview removed because the pull request was closed. 2024-07-09 14:27 UTC

[brauliorivas]

why we would treat them as undirected graphs?

You are right. I thought it could be useful anyway, because currently RecoParticles are being drawn randomly, so maybe by applying a similar idea as in the paper, we could draw these particles in such a position that is easy to "digest" (see connections with other particles, etc).

Specifically for this visualization, what I would do is the following

• Draw the MC graph
• Draw the ReconstructedParticle graph (from a user selected collection name)
• Use the MCRecoParticleAssociation collection (again user specified name) to draw links between the elements of the two graphs

Sorry, I should have fixed the test sooner so you can check this. But right now, all you mentioned is being drawn (user input is missing).

This should give us a version 0 of how this would look like to iterate on. It's probably quite horrible, but it serves as a basis :)

Yes, right now I draw an empty box for RecoParticle, but we can discuss what information to show.

• Add a toggle to "gray out" MCParticles and reco particles that have no counterpart in the other graph (maybe make this a selector where we can choose between: fully drawn, grayed out, fully removed)

Good, I'll add this. But when you mention that doesn't have a counterpart, it mean that either the MCParticle or RecoParticle don't have any MCRecoParticle association?

• Add the possibility to load a second MCRecoParticleAssociation collection to draw more connections (because it might be that the connections depend on whether you start on the MC side or on the reco side, even though the association does not necessarily have a direction). This one I would keep at low priority for the actual implementation, but make sure it's not impossible from a design point of view.

Well, in theory, when loading no matter what amount of MCRecoParticleAssociation collections are in the same event, all are going to be extracted and shown.

Having written down all that, it might be easiest to first make another PR (contrary to what I said during the meeting on Friday)

maybe we can use this one that already fixes some loading functionality.

where we only draw the reconstructed particles. To be most useful, this would probably require two main features. Drawing of either one or multiple reconstructed particle collections; Here one would be for PandoraPFOs or ReconstructedParticles as likely default names, but we can make the names user input. Multiple collections would most likely be the former, plus an additional Jets collection. Here the objects in the first collection are probably just a list, whereas, the objects of the Jets, will have so called constituents (stored in the particles field), that are objects of the first collection.

Could you please explain a bit more 😟. I thought users would only filter according to the names as in edm4hep (datatypes).

The second nice feature would then be, that if I click on one jet object, all the things that are not related to that are grayed out or hidden from the view entirely, but that can also go into another PR, once the basic displaying works.

Okay, then we could make it in another PR.

[tmadlener]

Thanks for fixing the tests, so that I can now look at a preview (instead of guessing, what will be drawn) :) with the renaming of dmx -> eede, the preview link for this PR point to the wrong place, but it's easy enough to fix, so we shouldn't bother doing anything about it, I think.

A few comments on the current state:

• The ReconstructedParticles are truly all over the place ;) I would have made a similar graph as the we do for the MCParticle, using the particle relation to draw (daughter) relations and then create the tree to view.
• Can we have two separate trees and then simply draw the associations between objects of the two trees "afterwards"?
• As far as I could see, this is simply drawn on top of the existing MCParticle view. Can we have a switch / toggle, to have that untouched and the new view as a separate option?

where we only draw the reconstructed particles. To be most useful, this would probably require two main features. Drawing of either one or multiple reconstructed particle collections; Here one would be for PandoraPFOs or ReconstructedParticles as likely default names, but we can make the names user input. Multiple collections would most likely be the former, plus an additional Jets collection. Here the objects in the first collection are probably just a list, whereas, the objects of the Jets, will have so called constituents (stored in the particles field), that are objects of the first collection.

Could you please explain a bit more 😟. I thought users would only filter according to the names as in edm4hep (datatypes).

What I mean with this is that there might be several different collections of type ReconstructedParticle in an event (the same is probably true for Track, Cluster, etc..). This is a bit different to the MCParticles, where there is usually only one collection. These collections all have different names, e.g. there might be a ReconstructedParticles collection and a Jets collection, both of type ReconstructedParticle. In this case we should let the user decide which collection to actually draw, because usually they will not want all of the ReconstructedParticles, but only some of them. Since the objects of different ReconstructedParticle collections can still be related to each other, this selection would have to allow to select multiple collection names. This would be the basic functionality, there then might be additional functionality that we can build on top of this.

-Add a toggle to "gray out" MCParticles and reco particles that have no counterpart in the other graph (maybe make this a selector where we can choose between: fully drawn, grayed out, fully removed)

Good, I'll add this. But when you mention that doesn't have a counterpart, it mean that either the MCParticle or RecoParticle don't have any MCRecoParticle association?

If you go through the MCRecoParticleAssociation they will always have a sim (pointing to a MCParticle) part and a rec (pointing to a ReconstructedParticle) part. There might be MCParticles and ReconstructedParticles in the original graphs of only those objects, which are not touched by any of these relations. But we can also add this toggle in a next PR, because otherwise this one will become very large again.

[brauliorivas]

I've implemented a "view" switcher as discussed in the last meeting. Users can switch between visualizing just MCParticle, RecoParticle, or a list-like visualization of both through MCRecoParticleAssociation. Now, this allows us to add any new view easily. However, there is a performance issue for MCRecoParticleAssociation, and I've not figured out what it is. Ironically, this issue is almost imperceptible if running on Firefox. (I've temporarily disabled filters because we should add filters according to the view).

Edit: The performance issue is when moving particles in the MCRecoParticleAssociation view.

[tmadlener]

Very nice. I just had a quick look at the preview. I think for the main functionality we can already start about finalizing this PR and then making some of the views that are currently here a bit nicer in follow-up PRs to not make this one too big. How hard would it be to bring back the Filter and PDG / symbol toggle for the MCParticle tree view? Can we tie this additional content to a given view?

Then already on the nitpicking side; Is it possible for the views to have "proper names", i.e. with spaces? I will try to have a look at the code a bit later as well.

[brauliorivas]

Hi @tmadlener, thank you so much for checking this today. Right now I've added back the filters only for the MCParticle tree view. It's not the best solution (just temporary) but works for now. I'll think of a way to add filters depending on the view better than now. Adding filters depending on the view is relatively easy, but the filter menu itself (adding, and removing elements) needs to be designed in another way. Also, I've added the full name depending on the view.

we can already start about finalizing this PR and then making some of the views that are currently here a bit nicer in follow-up PRs to not make this one too big.

Yeah, with the last changes, this is already good progress and works "well". Then we can continue with other objects/functionality.

[tmadlener]

Thanks a lot for bringing back the filters for the MCParticle tree. Currently it seems to be "sticky", i.e. if I switch to the MCParticle tree view once, it stays visible also if I switch back to another view, e.g. here for the PDG toggle.

Is it possible to fix this still with this PR? Or would that already involve the rework / redesign you are talking about?

[brauliorivas]

Is it possible to fix this still with this PR? Or would that already involve the rework / redesign you are talking about?

Yes, that would require me to redesign.

[brauliorivas]

Now that #48 is practically done, this PR may be closed as the other branch is a continuation of this one.

[tmadlener]

Is this PR effectively still its own feature? Then we could keep the split between this and #48, but if this PR doesn't stand on its own I am also fine with closing this and continuing with #48.

[brauliorivas]

Is this PR effectively still its own feature?

Well, #48 has the same work as here and fixes a few things (shows colors for objects and different properties for RecoParticle). So we can close this.