error=26, Text file busy

[yushkovskiy]

During run on CI I get the following error

[ERROR] Failed to execute goal com.github.os72:protoc-jar-maven-plugin:3.1.0.5:run (generate-test-protobuf-sources) on project commons-netty-rest-server: Unable to execute protoc-jar for /var/lib/teamcity/BuildAgents/buildAgent/work/756b569813502e3a/commons/commons-netty-rest/commons-netty-rest-server/src/test/proto/test.proto: Cannot run program "/var/lib/teamcity/BuildAgents/buildAgent/temp/buildTmp/protoc4150673099125997663.exe": error=26, Text file busy -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.github.os72:protoc-jar-maven-plugin:3.1.0.5:run (generate-test-protobuf-sources) on project commons-netty-rest-server: Unable to execute protoc-jar for /var/lib/teamcity/BuildAgents/buildAgent/work/756b569813502e3a/commons/commons-netty-rest/commons-netty-rest-server/src/test/proto/test.proto
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
    at org.apache.maven.lifecycle.internal.LifecycleThreadedBuilder$1.call(LifecycleThreadedBuilder.java:169)
    at org.apache.maven.lifecycle.internal.LifecycleThreadedBuilder$1.call(LifecycleThreadedBuilder.java:165)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.maven.plugin.MojoExecutionException: Unable to execute protoc-jar for /var/lib/teamcity/BuildAgents/buildAgent/work/756b569813502e3a/commons/commons-netty-rest/commons-netty-rest-server/src/test/proto/test.proto
    at com.github.os72.protocjar.maven.ProtocJarMojo.processFile(ProtocJarMojo.java:443)
    at com.github.os72.protocjar.maven.ProtocJarMojo.processTarget(ProtocJarMojo.java:391)
    at com.github.os72.protocjar.maven.ProtocJarMojo.performProtoCompilation(ProtocJarMojo.java:350)
    at com.github.os72.protocjar.maven.ProtocJarMojo.execute(ProtocJarMojo.java:297)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:106)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
    ... 11 more
Caused by: java.io.IOException: Cannot run program "/var/lib/teamcity/BuildAgents/buildAgent/temp/buildTmp/protoc4150673099125997663.exe": error=26, Text file busy
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
    at com.github.os72.protocjar.Protoc.runProtoc(Protoc.java:88)
    at com.github.os72.protocjar.Protoc.runProtoc(Protoc.java:61)
    at com.github.os72.protocjar.maven.ProtocJarMojo.processFile(ProtocJarMojo.java:436)
    ... 16 more
Caused by: java.io.IOException: error=26, Text file busy
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
    at java.lang.ProcessImpl.start(ProcessImpl.java:134)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
    ... 19 more

For mvn I use -T2C but I believe that it could be raise between different builds on the same agent. Also it could be result of JDK-8068370 (process) Executing recently modified file results in ETXTBSY

Is it possible to add some retry policy?

[os72]

Ack. What platform are you on? Does it fail each time?

This got reported once for Ubuntu, probably same JDK issue, apparently fixed https://github.com/os72/protoc-jar/issues/8

Yes we can try the retry approach

[yushkovskiy]

My get this error on my laptop about once a day:

$ uname -a
Linux yushkovskiy-nix 4.8.0-34-generic #36-Ubuntu SMP Wed Dec 21 17:24:18 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.10
Release:    16.10
Codename:   yakkety

$ java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

On CI hosts I see that about 5 times a day:

$ uname -a
Linux fol1-0805.search.yandex.net 3.18.35-35 #1 SMP Mon Jun 20 11:39:44 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 12.04.5 LTS
Release:    12.04
Codename:   precise

$ java -version
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

I not sure that reordering will help if that this is JDK bug. I guess setting executable flag in forked process could help

[os72]

OK. I'll need a day or two till I can look into it. I assume you're able to run snapshot versions?

[os72]

Could you try 3.1.0.6-SNAPSHOT? https://oss.sonatype.org/content/repositories/snapshots/com/github/os72/protoc-jar-maven-plugin/3.1.0.6-SNAPSHOT/

[yushkovskiy]

I failed to find com.github.os72:protoc-jar:jar:3.1.0.4-SNAPSHOT. Could you please upload it too?

[os72]

Done https://oss.sonatype.org/content/repositories/snapshots/com/github/os72/protoc-jar/3.1.0.4-SNAPSHOT/

You could also clone or fork the repos and build yourself, if changes are needed

[yushkovskiy]

Seems fixed, but I wish some more time to check it.

[yushkovskiy]

Looks like it fixed. Didn't reproduced on snapshot version

[os72]

Okay great, I'll make a release soon. Did you observe any retries?

[os72]

Released 3.2.0

[yushkovskiy]

Thanks! It is too many logs in our build and it is pretty hard to find something in it on CI.