search

daisy / pipeline-tasks

Used for project management of the DAISY Pipeline 2
Apache License 2.0
1 stars 0 forks source link

Custom probes for YourKit #26

Open bertfrees opened 9 years ago

bertfrees commented 9 years ago

YourKit allows you to hook in your own so called "probes" for monitoring events and recording information specific to your particular application. Information can be saved in a database that can be viewed in the user interface. See https://www.yourkit.com/docs/java/help/probes.jsp.

This way we can gather valuable information such as the names of XProc steps and XSLT sheets that are being loaded/invoced. I think I will create a new project in pipeline-build-utils named something like "yourkit-probes", with classes such as "org.daisy.pipeline.yourkit.probes.XProc" and "org.daisy.pipeline.yourkit.probes.XSLT", which can then be used as a utility in tests for example.

Any thought before I start?

josteinaj commented 9 years ago

Sounds great.

I've also found setting calabash logging to TRACE in etc/config-logging.xml to be very useful; you can determine which XSLT is invoked from there. It won't tell you about memory usage though, so if you hook into YourKit I think memory usage would be the most valuable information as that's not available from calabash profiliing or logging. Also, I don't know how to get detailed information about the XSLTs that are executed from XProc, such as how many times each template is invoked etc.

bertfrees commented 9 years ago

Yes I've also found the TRACE information of Calabash quite useful. Especially the intermediate documents that are displayed at the start of each p:for-each.

The YourKit probe gives us total freedom to record any information we want (well, if I understand correctly it's limited to arguments that are passed to methods) without having to modify any Calabash code. Also I like that you can view database tables in a UI instead of (or in addition to) looking at log files.

I haven't figured out how the gathered information can be linked with memory usage, hopefully that will become more clear when I'm using it.

rdeltour commented 9 years ago

SGTM. Do you think you can hook probes to e.g. XPath functions? (maybe via Saxon's API)

bertfrees commented 9 years ago

Yes, should be possible.

bertfrees commented 9 years ago

I've committed very basic probes for XPath, XSLT and XProc, basically a proof of concept. Feel free to improve them!