Overall Performance issues with Papyrus Modeling

[emammoser]

Issue and tracking information

Papyrus/CX as a whole feels slower than CX on RSA. No doubt these are significantly different tools at this point but our users have shown frustration with this. I would say on a whole, the median time to perform actions seems to be over 2x longer, especially as model size and complexity grows. This includes adding elements, renaming elements, modifying them, etc... Even just navigating in the model explorer. I am sure some of this slowdown impacts the model merging too. Are you guys aware of anything significantly different between RSA and Papyrus? They are both based on the Eclipse EMF, right? Do you know how much File IO might be occurring when modeling (before actually saving a model)?

In response to @ysroh comment about model opening,

One of the major slowdowns during the model open stage is to check model migration status, broken reference checks, and pathmap checks on the fly rather than via separate actions. They take up about 50% of the running time of model open action. We could improve slightly but not much as long as we want to check model sanity during the opening stage. I can profile the performance of other actions and will comment on the new ticket.

I would say I am less concerned about model opening, however, if we did have a dialog that showed the different stages like "Loading, performing reference check, checking migration status, etc.." this would ease our user's minds that Papyrus is doing something.

What I am more concerned about is the other actions like clicking around the model, creating new elements, new connectors, new diagrams, updating diagrams, etc... Another problem is that sometimes these operations are snappy, other times they take a few seconds, and other times they take a much longer amount of time.

I would love to provide you with reproducers of these problems, but at the moment, its hard to because they are so inconsistent. I think the best reproducer would just be a very complicated "nonsense" model, but we don't have one readily available. We may be able to create one, however, if we think we need one. I also want to point out that I am fairly confident a lot of these issues are because of eclipse and papyrus themselves, and not caused by CX. But I would love to see if we could find some improvements here. I will also note that the 2020 version of Papyrus does appear to be faster than the 2019 version, so I have noticed improvements there.

Developer's time Estimated effort to fix (hours):

Developer's Actual time spent on fix (hours)

Issue reporter to provide a detailed description of the issue in the space below

[ysroh]

@j26151 Is it possible to provide me with some more details or examples to reproduce? Most performance slowdown happens when the model is open or when classes are loaded for the first time. For example, when creating a new element for the first time after the tool is opened. These are normal slowdowns because Eclipse loads classes lazily and will disappear from subsequence calls.

I am looking for actions where a slowdown is detected each time the action is invoked, and I am having trouble reproducing it. Can you please test it with the latest update and help me to reproduce the issue? In the meantime, I will continue to use the profile tool to see if I see any hot spots in our code.

[ysroh]

Can you use the "papyrus_profile" branch to reproduce the problem? I've made some improvements by getting rid of calls to old Xtend to configure some elements and replacing them with advice bindings to the element types. It should be much faster to create some elements especially Deployment Plans, Arrays, etc.

[ysroh]

https://github.com/ZeligsoftDev/CX4CBDDS/pull/424

[emammoser]

Thank you @ysroh for your work here. I will try to get to testing this soon, but I can't make any promises.

[emammoser]

I have verified that the actions that you called out above are now significantly faster. This is a great start and hopefully we can identify other performance issues that you might be able to improve. I will try to play around with making and modifying assembly diagrams to see if there is anything there that is slow. A lot of the performance woes are always reported well after they occurred from our users so we don't have a lot to go on or to provide you specifics.

One thing that I have noticed is instantiating templated connectors appear to take a while. Especially the "DDS4CCM Connector Type" context menu option. Could you profile that next?

[ysroh]

Yes. I will do that. I suspect that the model merge is taking a long time when instantiating a template.

[emammoser]

The model merge? This shouldn't have anything to do with EMF Compare or model merging via. git?

[ysroh]

Sorry for the confusion. I meant the internal model merge which has nothing to do with the EMF model merge. The better words to describe it would be model copier with cross reference resolver.

[ysroh]

I tried to profile all other actions by creating the basic pubsub model from the scratch and I concluded that there are no other performance fixes are necessary. Instantiating the connector library takes longer because the tool needs to copy all elements from the template library, set all property values, and fix all references within the instantiated connector. I wasn't able to find any places where we could optimize.

[emammoser]

Some of our users have mentioned that instantiating a connector in 1.X with RSA was closer to instantaneous. I haven't verified that, but do you know why Papyrus would struggle so much more with this? At the end of the day I would expect the copy and updated references to be fast unless the individual operation of "updating a reference" to take 1 second, making 30 updates take 30 seconds.

[ysroh]

It seems to take about 4 secs to instantiate the DCPS connection library on my laptop which is running Core i5. Maybe I will try to profile the same action in RSA to compare.

[ysroh]

I couldn't attach the profiler to RSA, but the running time seems similar between the two. Using the same environment, RSA takes about 6 seconds to instantiate for the first time and 3 seconds afterwards. Papyrus takes about 7 seconds for the first time and 3.5 seconds afterwards. Papyrus is slightly slower, but I didn't find noticeable difference between the two.

[emammoser]

I am seeing the same behavior as you when instantiating the connector in a relatively small model like our BPS example. However, in a significantly larger model, I just tried twice and saw times around 25 seconds. This was on a build from the papyrus branch. I can install your papyrus_profile build and see if there is a difference but it appears that the size of the model affects the time it takes to create the connector.

[ysroh]

I see. I don't think the new branch will do anything. I will try to make a bigger model unless you can provide a reproducer in the private repo.

[ysroh]

Ok. I used the old ArtGallery model and it takes a lot longer there. I will investigate what's taking so long.

[emammoser]

I don't have one that I can send at the moment. At some point I would like to see us create a set of large models that have pathmaps between them so that we can reproduce these issues more easily. Our production models tend to have 50+ message definitions, 20+ components, and many assemblies. Then we have models that build off of these models.

What would be interesting is if we had a java API that we could tap into to make arbitrarily large models to profile. Such and api where I could call "create model(), create package(), instantiate connector(), addTypeToMessage()" etc... We could code something that could create multiple models of large sizes very quickly with dummy data. Just a thought at the moment, but it could be useful.

[emammoser]

Glad you were able to recreate with the ArtGallery

[ysroh]

I see the same slow running time on RSA using the ArtGallery model as well. The problem is not introduced in Papyrus. Nevertheless, I will try to debug this to see if I can improve the performance.

[ysroh]

Can you try it again to see if the performance issue is fixed?

[ysroh]

I found inefficient code related to the Deployment listener and improved it. Let me know if you find any other performance issues. Otherwise, I will close this issue once you verify the fix.

[emammoser]

Everything looks good here. I tried out as many random operations that I could think of. Specifically, for instantiating connectors in our production model, I was seeing 25 seconds before your fixes, and now I see 12s and 8s for 2 instantiations. This is much better than it was.

I am happy with the progress made here. Hopefully when we can release this to our users they won't have any more concerns and if they do we can try to nail down exact operations that were slow. I could not verify that CDP/CDD files generated without issue as it appears to be broken with assemblies with delegated ports. Since this branch didn't make anything visibly worse, and made many things better, I am okay with this being merged to the papyrus branch.

[ysroh]

Sounds good, thanks.

[ysroh]

Merged