Get statistics with Inespections

[Super8film87]

Hey,

I've trouble to find a solution to get statics of InspectionStatsE & InspectionStatsW. If I check following url

http://<teamcity>/app/rest/builds/id:<id>/statistics

I see the correct payload:

<properties count="16">
<script/>
<property name="ArtifactsSize" value="2242155"/>
<property name="BuildDuration" value="34495"/>
<property name="BuildDurationNetTime" value="33403"/>
<property name="buildStageDuration:artifactsPublishing" value="319"/>
<property name="buildStageDuration:buildFinishing" value="500"/>
<property name="buildStageDuration:buildStepRUNNER_33274" value="30902"/>
<property name="buildStageDuration:buildStepRUNNER_33275" value="2501"/>
<property name="buildStageDuration:firstStepPreparation" value="189"/>
<property name="buildStageDuration:toolsUpdating" value="1"/>
<property name="BuildTestStatus" value="1"/>
<property name="InspectionStatsE" value="11"/>
<property name="InspectionStatsW" value="6490"/>
<property name="queueWaitReason:Waiting_for_the_build_queue_distribution_process" value="78"/>
<property name="serverSideBuildFinishing" value="701"/>
<property name="SuccessRate" value="1"/>
<property name="TimeSpentInQueue" value="85"/>
</properties>

if I try to collect the data via velocity

#set ($statsValues = $build.statisticValues)
{
#foreach ($mapEntry in $statsValues.entrySet())
  #if ( $mapEntry.key == 'InspectionStatsE' || $mapEntry.key == 'InspectionStatsW')
    "$mapEntry.key": $mapEntry.value.intValue(),
  #end
#end
"statsValues" : "$statsValues",
## or like this

"getStatisticValues": "${build.getStatisticValues()}"
}

but this only responding:

"statsValues": "{buildStageDuration:buildStepRUNNER_29664=346475, buildStageDuration:artifactsPublishing=551, buildStageDuration:buildStepRUNNER_26288=10884, buildStageDuration:buildStepRUNNER_31602=536, serverSideBuildFinishing=48, buildStageDuration:buildStepRUNNER_31601=297, buildStageDuration:buildStepRUNNER_24038=539, buildStageDuration:sourcesUpdate=681, queueWaitReason:Waiting_for_the_build_queue_distribution_process=68, buildStageDuration:buildFinishing=710, buildStageDuration:toolsUpdating=1, buildStageDuration:firstStepPreparation=2, BuildDurationNetTime=358731}",
  "getStatisticValues": "{buildStageDuration:buildStepRUNNER_29664=346475, buildStageDuration:artifactsPublishing=551, buildStageDuration:buildStepRUNNER_26288=10884, buildStageDuration:buildStepRUNNER_31602=536, serverSideBuildFinishing=48, buildStageDuration:buildStepRUNNER_31601=297, buildStageDuration:buildStepRUNNER_24038=539, buildStageDuration:sourcesUpdate=681, queueWaitReason:Waiting_for_the_build_queue_distribution_process=68, buildStageDuration:buildFinishing=710, buildStageDuration:toolsUpdating=1, buildStageDuration:firstStepPreparation=2, BuildDurationNetTime=358731}"

Additional test:

#set ($InspectionStatsE_value = $build.getStatisticValue("InspectionStatsE"))
#set ($InspectionStatsW_value = $build.getStatisticValue("InspectionStatsW"))
{
"inpsectionError" : "$InspectionStatsE_value",
"inpsectionWarnings" : "$InspectionStatsW_value"
}

If I check on Preview Template Payload

I see correct payload

{
  "inpsectionError": "11",
  "inpsectionWarnings": "6490"
}

on endpoint-viewer:

{
  "inpsectionError": "$InspectionStatsE_value",
  "inpsectionWarnings": "$InspectionStatsW_value"
}

Can you provide some guidance - #https://javadoc.jetbrains.net/teamcity/openapi/current/jetbrains/buildServer/serverSide/SBuild.html#getFullStatistics() Do I check the wrong commands?

TeamCity Server: TeamCity Enterprise 2023.05.4 (build 129421) Plugin: 1.2.5

[netwolfuk]

Sorry for the delay. I'll look at this after I've fixed #236

[Super8film87]

No worries at all. Thx for the support

[Super8film87]

Any idea?

[netwolfuk]

Are you still running 2023.05.4 and tcWebhooks 1.2.5?

[Super8film87]

Yes we do - not sure when we will move the next version.

We have on Dev (2024.03) now

[netwolfuk]

Is this the result of an actual build sending to the /webhooks/endpoint.html ? Or was it from a "test" from the template preview?

on endpoint-viewer:

{
"inpsectionError": "$InspectionStatsE_value",
"inpsectionWarnings": "$InspectionStatsW_value"
}

[Super8film87]

no to /webhooks/endpoint.html

[netwolfuk]

I am wondering if inspections are assembled after the build completes, and perhaps are not available at the time that the webhook runs. None of my test builds appear to have inspections on them in the REST API or in webhook payload. I am not using a jetbrains inspections tool. Mine are just standard maven builds.

[Super8film87]

Hm could be a possible. For unit tests it is working - I'll check with a small example.

[Super8film]

Created an issue here

[netwolfuk]

Thanks

[Super8film]

From the issue above:

Hello folks,

The value for InspectionStatsE and InspectionStatsW parameters is calculated and published during TeamCity's event buildFinished().

The same listener is used for WebHookListener and other listeners.

It could happen that plugin's buildFinished() is exectuted before TeamCity's processing of InspectionStats* metrics.

The solution could be to implement PositionAware interface in WebHookListener and set constraint to PositionConstraint.last(), so the web hook is executed after all changes made by TeamCity itself. @Net Wolf could you do this change in the plugin?

Kind regards,
KIR

can you provide feedback?

[netwolfuk]

I'm so sorry I missed that comment. I'm looking at this today. I'll get a build ready for you to test by tomorrow.

[netwolfuk]

I have implemented the change as suggested by Kir. Can you please test a build from this branch?

You can just login as guest download the zip.

I've built this from version 2.0.1. Are you still on 1.2.5? If so, I'll need to do a new build for you.

[Super8film]

We are using 2.0.1 => will come to with feedback. Thx for this fast feedback.

[Super8film]

Sorry for the delay -

using: /plugins/tcWebHooks-2.0.1-build.506.zip

same issue to report. I dont see it correct in the endpointviewer :(

[Super8film87]

did already add the change proposed by jet brains?

[netwolfuk]

The change ended up being a significant amount of work. Can you please see https://github.com/tcplugins/tcWebHooks/wiki/Waiting-for-Build-Statistics-to-be-published and download the latest build from issue_235-missing_inspections

[netwolfuk]

I have pushed 2.0.2-rc.2 The build is now available at issue_235-missing_inspections

This fixes an issue where it would take an hour to time out when not all build statistics had been received even though the build had completed and everything else had worked correctly. Now has another setting that allows controlling the buildStatisticsBuildCompletedTimeout either from the main-config.xml or by overriding in a webhook parameter.

See updated docs at: Waiting-for-Build-Statistics-to-be-published

[netwolfuk]

I've updated the docs with a diagram to better explain the timeline and which configuration settings affect the execution of the webhook.