allure-framework / allure-gradle

Allure Gradle Plugin
Apache License 2.0
83 stars 36 forks source link

In Allure 2.9.3 the ability to specify allure-results folder (resultsDir) has disappeared. #72

Open rickflar opened 2 years ago

rickflar commented 2 years ago

I'm submitting a ...

What is the current behavior?

Right now, i don't have an ability to specify custom allure-results folder, using resultsDir variable.

What is the expected behavior?

Have an ability to specify custom allure-results folder, like it was in allure-gradle 2.8.1.

Please tell us about your environment:

Allure version 2.14.0
Test framework TestNG
Generate report using allure-gradle@2.9.3
vlsi commented 2 years ago

Why do you need it?

rickflar commented 2 years ago

Because it used to be in earlier versions - we used it to set up a custom folder for the results of the run.

vlsi commented 2 years ago

The thing is there might be mulitple tasks (e.g. test, integrationTest, javaexec, etc) that collect Allure raw results. That makes "single result folder" complicated.

vlsi commented 2 years ago

For instance, https://docs.gradle.org/current/userguide/more_about_tasks.html#sec:task_input_output_parallelism

Gradle will inspect the outputs of tasks when selecting the next task to run and will avoid concurrent execution of tasks that write to the same output directory

baev commented 2 years ago

@vlsi can we provide a possibility to override output per task?

vlsi commented 2 years ago

I would like to see an end-to-end use-case

rickflar commented 2 years ago

As i noticed, results appear in allure-results folder in build folder by default. So, it's the case of "single result folder", isn't it? Or without setting a resultsDir variable, results can appear in different folders?

For instance, https://docs.gradle.org/current/userguide/more_about_tasks.html#sec:task_input_output_parallelism

Gradle will inspect the outputs of tasks when selecting the next task to run and will avoid concurrent execution of tasks that write to the same output directory

vlsi commented 2 years ago

The folder is not documented, so it can be changed.

admizh commented 2 years ago

In this case - @Rickflar if you need to use a working plugin with task customization, I mean - override default report outputs, disabling SPI OFF (), task aggregation - just not use the latest versions for Allure. Unfotunantliy you lose support for gradle 7 - because the last plugin changes/fixes not working in multiple places, not support manual customizations.

Currently, all work as expected with old versions of allure gradle plugin 2.8.1, gradle 6.9.1:

id("io.qameta.allure") version "2.8.1"
tasks.withType<Wrapper> {
  gradleVersion = "6.9.1"
  distributionType = Wrapper.DistributionType.ALL
}

@vlsi in my case (and many allure users) - we did not use autoconfiguration {} closure at all - we use allure extensions just to have proper API binding from gradle to allure command-line. Like this:

configure<AllureExtension> {
  version = `allure-version`
  autoconfigure = false
  aspectjweaver = false
  configuration = "testImplementation"
  reportDir = file(dynamic["allure.results.directory"].toStringOrEmpty())
}

val aggregatedReport by tasks.creating(AllureReport::class) {
  group = "e2e-allure"
  clean = true
  reportDir = file("${rootProject.buildDir}/allure-report")
  resultsDirs = subprojects.map {
    file("${it.buildDir}/allure-results")
    file("${it.projectDir}/allure-results")
  }.filter { it.exists() }
  resultsDirs.add(file("${rootProject.buildDir}/allure-results"))
}

tasks.withType<io.qameta.allure.gradle.task.AllureServe> {
  group = "e2e-allure"
  dependsOn("downloadAllure")
}

tasks.withType<io.qameta.allure.gradle.task.DownloadAllure> {
  group = "e2e-allure"
}

tasks.withType<AllureReport> {
  if (this.name == "allureReport") {
    this.enabled = false
    group = "deprecated"
  }
}
admizh commented 2 years ago

@baev maybe you had the same case as mine in the previous comment? Can you help and advise us here? ... how we can create a correct working example with the latest gradle API + not-broken plugin. theoretically, I can just make direct calls to bat files...

jacksonwen001 commented 2 years ago

Does there are any solutions? We have this problem, I running test case in different environments but I wanna keep different results in different folder. such as if I specify -Denv=qa and results will place at build/projectName/qa/allure-results and I specify -Denv=prod and results will place at build/projectName/prod/allure-results. and the same as report. Any solutions can help?

Leo-Lem commented 2 years ago

Has anyone found a solution to this problem? We're facing the same issue here...

vlsi commented 2 years ago

@Leo-Lem , would you please clarify what is your issue exactly? What is your use case, and why do you need to customize the folder?

Leo-Lem commented 2 years ago

Hey @vlsi, we are updating a mobile test automation framework with appium, gradle, testng, allure etc. Due to parallel test execution, we'd like to have separate "build" directories for each gradle build to avoid conflicts (in the format of 'out-timestamp' under a results directory). This works well enough (by specifying the project's buildDir in the build.gradle file), only the allure-results are always generated in root/build/allure-results.

We'd appreciate it if a custom allure results directory could be configured...

re-l124c41 commented 1 year ago

In my case allure just fails to produce raw results with error: io.qameta.allure.AllureResultsWriteException: Could not write Allure test result container caused by FileNotFoundException. Tried to setup results dir in allure.properties but error remains the same. While debugging allure code i noticed that this.mapper.writeValue(file.toFile(), testResultContainer); fails if called in eval window as-is, but if a call it with file, resolved to absolute path: this.mapper.writeValue(new File(file.toAbsolutePath().toString), testResultContainer); it works just fine. If I change path to directory in allure.properties to absolute path it works too.

nicolabeghin commented 9 months ago

Any update on this? thanks!

re-l124c41 commented 7 months ago

Ended up using somewat hacky solution. It works with gradle 8.5, java 21, allure 2.25.0

  1. Add file src/test/resources/allure.properties.bak with text:
    allure.results.directory=RESULT_DIR
  2. In build.gradle add task and configure task dependency:
    
    task allureProperties(type: Copy) {
    outputs.upToDateWhen{ false }
    from file('src/test/resources/allure.properties.bak')
    into 'src/test/resources/'
    rename{ 'allure.properties' }
    filter{ line -> line.replace('RESULT_DIR', file(layout.buildDirectory.dir('allure-results')).toString()) }
    }

processTestResources{ dependsOn(tasks.allureProperties) }

This task will copy content of bakup properties file to actual file and replace results directory value with absolute path.

If you don't care about other properties and need to set only result path, you can omit bakup file and simplify tasks to:

task allureProperties { outputs.upToDateWhen{ false } doFirst { def props = file('src/test/resources/allure.properties') props.write("allure.results.directory=${file(layout.buildDirectory.dir('allure-results'))}") } } processTestResources{ dependsOn(tasks.allureProperties) }

nicolabeghin commented 7 months ago

Ended up using somewat hacky solution. It works with gradle 8.5, java 21, allure 2.25.0 If you don't care about other properties and need to set only result path, you can omit bakup file and simplify tasks to:

task allureProperties {
   outputs.upToDateWhen{ false }
   doFirst {
      def props = file('src/test/resources/allure.properties')
      props.write("allure.results.directory=${file(layout.buildDirectory.dir('allure-results'))}")
   }
}
processTestResources{
   dependsOn(tasks.allureProperties)
}

thanks a lot, hacky but does the trick!! kudos for coming up with that :D