search

linkedin / linkedin-gradle-plugin-for-apache-hadoop

Apache License 2.0
117 stars 76 forks source link

Add a workaround to handle visiting files multiple times when merging tempprops into .flow files. #249

Closed ypadron-in closed 5 years ago

ypadron-in commented 5 years ago

The visit function from Gradle in HadoopZipExtension.mergeTempPropsWithFlowFiles() visits files multiple times. This causes the error below when trying to create a merge file it already created on a previous visit to the same file and marked it as read only.

Caused by: java.io.FileNotFoundException:/Users/ypadron/azkaban/myProject/aFlow.flow (Permission denied)
        at com.linkedin.gradle.zip.YamlMerge.merge(YamlMerge.groovy:42)
        at com.linkedin.gradle.zip.YamlMerge$merge$0.call(Unknown Source)
        at com.linkedin.gradle.zip.HadoopZipExtension$_mergeTempPropsWithFlowFiles_closure4.doCall(HadoopZipExtension.groovy:247)
        ... 55 more

Because the root cause of the issue is in code we don’t control the best option is to implement a workaround to handle the multiple visits.

jamiesjc commented 5 years ago

Thanks for the fix. Please also bump gradle.properties to next version and update VERSION.md and CONTRIBUTORS.md files.