jenkinsci / peass-ci-plugin

Jenkins plugin for peass to support performance measurement in CI
https://plugins.jenkins.io/peass-ci/
GNU Affero General Public License v3.0
4 stars 12 forks source link

Jenkins RTS and measurement logs are accumulating #226

Closed alex-iotiq closed 1 year ago

alex-iotiq commented 1 year ago

When executing cleanPerformanceMeasurement() and RTS in the same Jenkins job multiple times, the RTS and measurement logs are accumulating.

For example

  1. Build

    measurement VM 0 log:

    com.example.android_example.LogHistoryInstrumentedTest:.
    
    Time: 0.453
    
    OK (1 test)
  2. Build

    measurement VM 0 log:

    com.example.android_example.LogHistoryInstrumentedTest:.
    
    Time: 0.453
    
    OK (1 test)
    
    com.example.android_example.LogHistoryInstrumentedTest:.
    
    Time: 0.472
    
    OK (1 test)

Build 2 now has the log from build 1 and 2.

Looking back in the log of build 1 you can find both results of build 1 and 2.

Pipeline looks like this.

pipeline {
  agent any
  stages {
    stage('Clone android-example-project') {
      steps{
          git branch: 'emulated_test', url: 'https://github.com/alex-iotiq/android-example-2.git'
        }
      }
    stage('Clean old Peass-ci data'){
      steps{
        cleanPerformanceMeasurement cleanMeasurement: true, cleanRCA: true, cleanRTS: true
      }
    }
    stage('Peass'){
      steps{
          measure VMs: 2, iterations: 1, repetitions: 1, warmup: 0,
          generateCoverageSelection: true, 
          executeRCA: false,
          nightlyBuild: false, 
          redirectSubprocessOutputToFile: false, redirectToNull: false,

          useAnbox: true,
          testExecutor: 'de.dagere.peass.execution.gradle.AnboxTestExecutor',
          testTransformer: 'de.dagere.peass.testtransformation.JUnitTestTransformer',
          testGoal: 'testRelease', 
          testClazzFolders: 'src/androidTest/java:src/androidTest', 
          androidMinSdkVersion: "26",
          androidTargetSdkVersion: "29"
      }
    }
  }
}
DaGeRe commented 1 year ago

Thank you for the issue, unfortunately, I cannot reproduce this.

Using the following pipeline:

pipeline {
  agent any
  stages {
    stage('Clone android-example-project') {
      steps{
          git branch: 'main', url: 'git@github.com:DaGeRe/demo-project-parameterized.git'
        }
      }
    stage('Clean old Peass-ci data'){
      steps{
        cleanPerformanceMeasurement cleanMeasurement: true, cleanRCA: true, cleanRTS: true
      }
    }
    stage('Peass'){
      steps{
          measure VMs: 2, iterations: 1, repetitions: 1, warmup: 0,
          generateCoverageSelection: true, 
          executeRCA: false,
          nightlyBuild: false, 
          redirectSubprocessOutputToFile: false, redirectToNull: false
      }
    }
  }
}

works fine, and if I execute it twice, the logs appear only once.

Can you confirm this, or am I maybe executing it somehow differently than you?

alex-iotiq commented 1 year ago

Thanks for your reply. I've tested my pipeline again and it seems like it resolved itself. I've tried more configurations but couldn't reproduce it again. It must have been a problem from past versions, since I'm using newer versions.