Gradle properties in config values are not expanded correctly in generateVoiceConfig

[psibre]

We can probably solve this by using an Ant Concat task with filtering...

[psibre]

A quick and dirty WiP PoC:

build.gradle

plugins {
    id 'base'
}

apply plugin: BarPlugin

foobar = 'baz'

class FooPlugin implements Plugin<Project> {

    @Override
    void apply(Project project) {
        project.ext.foobar = 'foobar'
        def fooTemplateTask = project.tasks.create('createFooTemplate', TemplateTask) {
            destFile.set(project.layout.buildDirectory.file('foo.txt'))
        }
        project.tasks.create('filterTemplates', FilterTask) {
            srcFiles.add(fooTemplateTask.destFile)
            destFile.set(project.layout.buildDirectory.file('filtered.txt'))
        }
    }
}

class TemplateTask extends DefaultTask {

    @OutputFile
    final RegularFileProperty destFile = project.objects.fileProperty()

    @TaskAction
    void template() {
        destFile.get().asFile.withWriter { writer ->
            writer.println "$name = \$foobar"
        }
    }
}

class FilterTask extends DefaultTask {

    @InputFiles
    final ListProperty<RegularFile> srcFiles = project.objects.listProperty(RegularFile)

    @OutputFile
    final RegularFileProperty destFile = project.objects.fileProperty()

    @TaskAction
    void filter() {
        ant.properties.foobar = project.findProperty('foobar')
        ant.concat(destfile: destFile.get()) {
            filelist(files: srcFiles.get().asList().join(','))
            filterchain {
                expandproperties {
                    propertyset {
                        propertyref(name: 'foobar')
                    }
                }
            }
        }
    }
}

class BarPlugin implements Plugin<Project> {

    @Override
    void apply(Project project) {
        project.pluginManager.apply(FooPlugin)
        def barTemplateTask = project.tasks.create('createBarTemplate', TemplateTask) {
            destFile.set(project.layout.buildDirectory.file('bar.txt'))
        }
        project.tasks.findByName('filterTemplates').configure {
            srcFiles.add(barTemplateTask.destFile)
        }
    }
}

However, I cannot figure out how to make the expandproperties work as expected, let alone dynamically mapping Gradle's project.properties...

[psibre]

So let's go with Gradle-based property expansion, or a GroovyTemplateEngine and avoid getting to grips with the Ant/Gradle property mapping.

[seblemaguer]

ok, for now as a patchy solution which works I replace the generateVoiceConfig task action by

        def engine = new groovy.text.SimpleTemplateEngine()                                                                                                                                                                       
        def bindings = ["project" : project]                                                                                                                                                                                      
        def tmp_config = [:]                                                                                                                                                                                                      
        tmp_config["name"] = project.marytts.voice.name                                                                                                                                                                           
        tmp_config["locale"] = project.marytts.voice.maryLocale                                                                                                                                                                   
        tmp_config["${voiceType()}.voices.list"] = project.marytts.voice.name                                                                                                                                                     

        ([                                                                                                                                                                                                                        
            domain      : 'general',                                                                                                                                                                                              
            gender      : project.marytts.voice.gender,                                                                                                                                                                           
            locale      : project.marytts.voice.locale,                                                                                                                                                                           
            samplingRate: project.marytts.voice.samplingRate                                                                                                                                                                      
        ] + config.get()).each { key, value ->                                                                                                                                                                                    
            tmp_config["voice.${project.marytts.voice.name}.$key"] = engine.createTemplate(value.toString()).make(bindings)                                                                                                       
        }                                                                                                                                                                                                                         

        project.marytts.component.config = tmp_config                                                                                                                                                                                                                                   

it is working and do what we want. At Least I can progress with this on my side and you have time to beautifying the thingy after.

[psibre]

Update, here's an MWE with Gradle-based template filtering and low-level Groovy-Reader/Writer-based concatenation. Needs two hops (filter templates into a temp dir, then concat from there into the destination file), but it works:

plugins {
    id 'base'
}

apply plugin: BarPlugin

foobar = 'baz'

class FooPlugin implements Plugin<Project> {

    @Override
    void apply(Project project) {
        project.ext.foobar = 'foobar'
        def fooTemplateTask = project.tasks.create('createFooTemplate', TemplateTask) {
            destFile.set(project.layout.buildDirectory.file('foo.txt'))
        }
        project.tasks.create('filterTemplates', FilterTask) {
            srcFiles.add(fooTemplateTask.destFile)
            destFile.set(project.layout.buildDirectory.file('filtered.txt'))
        }
    }
}

class TemplateTask extends DefaultTask {

    @OutputFile
    final RegularFileProperty destFile = project.objects.fileProperty()

    @TaskAction
    void template() {
        destFile.get().asFile.withWriter { writer ->
            writer.println "$name = \$foobar"
        }
    }
}

class FilterTask extends DefaultTask {

    @InputFiles
    final ListProperty<RegularFile> srcFiles = project.objects.listProperty(RegularFile)

    @OutputFile
    final RegularFileProperty destFile = project.objects.fileProperty()

    @TaskAction
    void filter() {
        project.copy {
            from srcFiles
            into temporaryDir
            expand project.properties
        }
        destFile.get().asFile.withWriter { writer ->
            project.fileTree(temporaryDir).each { srcFile ->
                writer << srcFile.newReader()
            }
        }
    }
}

class BarPlugin implements Plugin<Project> {

    @Override
    void apply(Project project) {
        project.pluginManager.apply(FooPlugin)
        def barTemplateTask = project.tasks.create('createBarTemplate', TemplateTask) {
            destFile.set(project.layout.buildDirectory.file('bar.txt'))
        }
        project.tasks.findByName('filterTemplates').configure {
            srcFiles.add(barTemplateTask.destFile)
        }
    }
}

[psibre]

The reason this bug was missed in the plugin's functional tests is that we don't actually override or add any custom entries to the generated config in the tests. But this is exactly what all of the voices do to customize resources, e.g., in marytts/voice-cmu-slt's build.gradle:

generateVoiceConfig {
    afterEvaluate {
        config << [
                targetCostWeights: "jar:/marytts/voice/$voice.nameCamelCase/customTargetCostWeights.txt",
                joinCostWeights  : "jar:/marytts/voice/$voice.nameCamelCase/customJoinCostWeights.txt",
        ]
    }
}

So first, we need to expand our test coverage to reproduce this bug.

[psibre]

Upon closer inspection, it's not so much the custom entries that are the trigger for this bug going unnoticed, but the fact that the test that covers this was disabled in 376dc156fac83c716cb63b726e4e126751d82119!

[psibre]

This is turning in to quite an endeavor... I think we need to do another MR to follow up #129 and ensure that all tests are active and passing. Several things are not working as expected, and it essentially boils down to how the voicebuilding plugin uses (or doesn't use) the component plugin to generate the voice config.