search

gradle / gradle-native

The home of Gradle's support for natively compiled languages
https://blog.gradle.org/introducing-the-new-cpp-plugins
Apache License 2.0
92 stars 8 forks source link

Provide diagnostic metrics of project lock wait time #902

Closed adammurdoch closed 5 years ago

adammurdoch commented 5 years ago

It's likely that the stricter project access will lead to performance regressions in certain builds. We should add some internal metrics of the time spent waiting on project locks during a build to help diagnose these.

Could be an internal flag or some debug logging or whatever.

ghale commented 5 years ago

I took a first (simple) stab at this here: https://github.com/gradle/gradle/pull/7431

This basically gathers up the total accumulated time that stuff is waiting on project locks. This might be ok for measuring improvement, but could be misleading for insight - for instance, if it reports 4000ms, that doesn't mean we wasted 4000ms of the total build time waiting for locks, it could be 2 tasks blocked concurrently on the same project for 2000ms or 40 tasks blocked for 100ms.

We should think about what information might be more informative/diagnostic here. For instance, would it make sense to have this broken down by project? Or should we try to measure "real" waiting time instead of total time (i.e. 2 tasks blocked concurrently for 1s is measured as 1s, not 2s)?

big-guy commented 5 years ago

Are there any other kind of locks we might hold that should be tracked too?

Would a counter on the number of lock/unlock actions that occur on a per-thread? per-operation? per-something be a useful metric too? e.g., if when executing a task in a single project build, it would be weird to see 1000's of lock/unlocks.

Could we wrap this into build operations only when the flag is enabled so we could use the operation trace to see these as separate events and attach things like number of things waiting?

ghale commented 5 years ago

Yeah, I think we eventually want to go the build operation route with this. I kind of held off on making this one simply because I was thinking a simple metric that could be compared across changes would be more useful in the short term. But overall, I definitely think having an operation that could be visualized and a "what's waiting on me?" metric would be immensely helpful.