search

fabric8io / docker-maven-plugin

Maven plugin for running and creating Docker images
https://dmp.fabric8.io
Apache License 2.0
1.88k stars 643 forks source link

Parallel maven build warnings #905

Open alexandrvb opened 6 years ago

alexandrvb commented 6 years ago

Description

I've got following warnings when trying to use docker-maven-plugin in parallel mode (mvn docker:build -T):

[WARNING] *****************************************************************
[WARNING] * Your build is requesting parallel execution, but project      *
[WARNING] * contains the following plugin(s) that have goals not marked   *
[WARNING] * as @threadSafe to support parallel building.                  *
[WARNING] * While this /may/ work fine, please look for plugin updates    *
[WARNING] * and/or request plugins be made thread-safe.                   *
[WARNING] * If reporting an issue, report it against the plugin in        *
[WARNING] * question, not against maven-core                              *
[WARNING] *****************************************************************
[WARNING] The following plugins are not marked @threadSafe in project:
[WARNING] io.fabric8:docker-maven-plugin:0.23.0
[WARNING] Enable debug to see more precisely which goals are not marked @threadSafe.
[WARNING] *****************************************************************

Info

wborn commented 6 years ago

Even though there is a warning we've been using it to build containers without any problems in parallel builds for months now. :slightly_smiling_face:

yufei-cai commented 6 years ago

I'm building docker images in parallel from multiple modules independent of each other. Docker-maven-plugin 0.23.0 hangs at the 'build' goal in maven 3.5.2, but it runs through without problems in maven 3.3.9. Does anyone want to work on this and need a reproducer?

rhuss commented 6 years ago

@yufei-cai it would be awesome if you could send in a stack trace when it hangs (CTRL-\ or 'kill -QUIT' to the Maven process).

Alternatively, if you can send in a sample script that would be very helpful, too.

Thanks!

ffendt commented 6 years ago

@rhuss : on behalf of @yufei-cai i'm sending in a threaddump.log.

rhuss commented 6 years ago

@ffendt Thanks ! I'm currently in christmas break but will have a look asap.

burtsevyg commented 5 years ago

maven 3.5.4 dmp 0.27.2 openjdk 8 191 mvn -T 1C -B -Dmaven.test.failure.ignore clean deploy

build and push goal doesn't executed

            <plugin>
                <groupId>io.fabric8</groupId>
                <artifactId>docker-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>docker-build-and-push-${project.artifactId}</id>
                        <phase>verify</phase>
                        <goals>
                            <goal>build</goal>
                            <goal>push</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
burtsevyg commented 5 years ago

@rohanKanojia Could you comment on #905 please. Is it possible to resolve it?

mhermosi commented 5 years ago

Versions and execution command:

Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T14:33:14-04:00)
Maven home: /Users/user/Development/maven
Java version: 12.0.1, vendor: Oracle Corporation, 
runtime: /Library/Java/JavaVirtualMachines/openjdk-12.0.1.jdk/Contents/Home

command:
mvn -B -s ./settings.xml -Dmaven.test.skip=true clean install docker:build docker:push -P private-registry -T 1C

I have another thing about parallel build. When using -T option on maven and on single or multiple executions the plugin lose the defined properties:

${container.registry}, ${image.prefix} and ${project.version}

<plugin>
   <groupId>io.fabric8</groupId>
      <artifactId>docker-maven-plugin</artifactId>
      <executions>
            <execution>
                  <id>demo-ui-build</id>
                  <configuration>
                        <registry>${container.registry}</registry>
                        <images>
                           <image>
                              <name>${image.prefix}/${project.artifactId}:${project.version}</name>
                              <build>
                                 <dockerFileDir>${project.basedir}</dockerFileDir>
                                 <dockerFile>Dockerfile</dockerFile>
                              </build>
                           </image>
                     </images>
                  </configuration>
            </execution>
            <execution>
                  <id>app-ui-build</id>
                  <configuration>
                        <registry>${container.registry}</registry>
                        <images>
                           <image>
                              <name>${image.prefix}/app-ui:${project.version}</name>
                              <build>
                                 <dockerFileDir>${project.basedir}</dockerFileDir>
                                 <dockerFile>Dockerfile</dockerFile>
                              </build>
                           </image>
                     </images>
                  </configuration>
            </execution>
         </executions>
  </plugin>

For some reason ${project.artifactId} is not lost.... but ${project.version} is lost and seems to be defaulted to latest

The weird thing is that ${image.prefix} seems to be getting the last part of the parent pom.xml groupId I mean.. if artifact is something like com.domain.xxx.yyy:artifactName:0.0.1-SNAPSHOT and ${image.prefix} = base then the result of image name is yyy/artifactName:latest instead of base/artifactName:0.0.1-SNAPSHOT

${container.registry} is being set on the profile definition.

In conclusion, I think this plugin should review its coding to support thread-safety so we can use the whole potential of

mhermosi commented 5 years ago

Found the issue.

I was executing maven-exec-plugin on package phase and also docker plugin on package phase. I moved maven-exec-plugin to compile (because I am doing some javascript build) and docker to install and seems to be working fine ... even for parallel build

mattnelson commented 5 years ago

@rhuss Here is a stack during a parallel build when it appears to be hung, locking on object 0x0000000782adfa28

Culprit in maven https://github.com/apache/maven/blob/maven-3.6.1/maven-model-builder/src/main/java/org/apache/maven/model/interpolation/StringSearchModelInterpolator.java#L296

Based on the change log for 3.6.2 I believe this issue may be addressed. https://issues.apache.org/jira/issues/?jql=project%20%3D%20MNG%20AND%20fixVersion%20%3D%203.6.2

thread with the lock

"BuilderThread 1" #27 prio=5 os_prio=31 cpu=15490.35ms elapsed=90.94s tid=0x00007febea75c800 nid=0x730f runnable  [0x000070000f068000]
   java.lang.Thread.State: RUNNABLE
    at java.lang.Class.isAssignableFrom(java.base@11.0.4/Native Method)
    at jdk.internal.reflect.UnsafeFieldAccessorImpl.ensureObj(java.base@11.0.4/UnsafeFieldAccessorImpl.java:57)
    at jdk.internal.reflect.UnsafeObjectFieldAccessorImpl.get(java.base@11.0.4/UnsafeObjectFieldAccessorImpl.java:36)
    at java.lang.reflect.Field.get(java.base@11.0.4/Field.java:418)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction$MapField.doInterpolate(StringSearchModelInterpolator.java:430)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction$CacheField.interpolate(StringSearchModelInterpolator.java:302)
    - locked <0x0000000782adfa28> (a java.lang.reflect.Field)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction$CacheItem.interpolate(StringSearchModelInterpolator.java:275)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction.traverseObjectWithParents(StringSearchModelInterpolator.java:160)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction.run(StringSearchModelInterpolator.java:134)
    at java.security.AccessController.doPrivileged(java.base@11.0.4/Native Method)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator.interpolateObject(StringSearchModelInterpolator.java:81)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator.interpolateModel(StringSearchModelInterpolator.java:64)
    at org.apache.maven.model.building.DefaultModelBuilder.interpolateModel(DefaultModelBuilder.java:781)
    at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:388)
    at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:171)
    at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:340)
    at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:297)
    at org.apache.maven.project.artifact.MavenMetadataSource.retrieveRelocatedProject(MavenMetadataSource.java:577)
    at org.apache.maven.project.artifact.MavenMetadataSource.retrieve(MavenMetadataSource.java:190)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:551)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.collect(DefaultLegacyArtifactCollector.java:148)
    at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:504)
    at org.apache.maven.repository.legacy.LegacyRepositorySystem.resolve(LegacyRepositorySystem.java:367)
    at org.apache.maven.plugins.assembly.artifact.DefaultDependencyResolver.resolveTransitively(DefaultDependencyResolver.java:233)
    at org.apache.maven.plugins.assembly.artifact.DefaultDependencyResolver.resolve(DefaultDependencyResolver.java:119)
    at org.apache.maven.plugins.assembly.artifact.DefaultDependencyResolver.resolveDependencySets(DefaultDependencyResolver.java:155)
    at org.apache.maven.plugins.assembly.archive.phase.DependencySetAssemblyPhase.execute(DependencySetAssemblyPhase.java:95)
    at org.apache.maven.plugins.assembly.archive.DefaultAssemblyArchiver.createArchive(DefaultAssemblyArchiver.java:181)
    at io.fabric8.maven.docker.assembly.DockerAssemblyManager.createAssemblyArchive(DockerAssemblyManager.java:466)
    at io.fabric8.maven.docker.assembly.DockerAssemblyManager.createDockerTarArchive(DockerAssemblyManager.java:115)
    at io.fabric8.maven.docker.service.ArchiveService.createArchive(ArchiveService.java:123)
    at io.fabric8.maven.docker.service.ArchiveService.createArchive(ArchiveService.java:118)
    at io.fabric8.maven.docker.service.BuildService.buildImage(BuildService.java:132)
    at io.fabric8.maven.docker.service.BuildService.buildImage(BuildService.java:70)
    at io.fabric8.maven.docker.BuildMojo.buildAndTag(BuildMojo.java:74)
    at io.fabric8.maven.docker.BuildMojo.processImageConfig(BuildMojo.java:105)
    at io.fabric8.maven.docker.BuildMojo.executeInternal(BuildMojo.java:61)
    at io.fabric8.maven.docker.AbstractDockerMojo.execute(AbstractDockerMojo.java:230)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:202)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:198)
    at java.util.concurrent.FutureTask.run(java.base@11.0.4/FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call(java.base@11.0.4/Executors.java:515)
    at java.util.concurrent.FutureTask.run(java.base@11.0.4/FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.4/ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.4/ThreadPoolExecutor.java:628)
    at java.lang.Thread.run(java.base@11.0.4/Thread.java:834)

thread waiting on the lock

"BuilderThread 23" #283 prio=5 os_prio=31 cpu=4023.80ms elapsed=83.59s tid=0x00007febf194d800 nid=0xc91f waiting for monitor entry  [0x000070000f779000]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction$CacheField.interpolate(StringSearchModelInterpolator.java:296)
    - waiting to lock <0x0000000782adfa28> (a java.lang.reflect.Field)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction$CacheItem.interpolate(StringSearchModelInterpolator.java:275)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction.traverseObjectWithParents(StringSearchModelInterpolator.java:160)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator$InterpolateObjectAction.run(StringSearchModelInterpolator.java:134)
    at java.security.AccessController.doPrivileged(java.base@11.0.4/Native Method)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator.interpolateObject(StringSearchModelInterpolator.java:81)
    at org.apache.maven.model.interpolation.StringSearchModelInterpolator.interpolateModel(StringSearchModelInterpolator.java:64)
    at org.apache.maven.model.building.DefaultModelBuilder.interpolateModel(DefaultModelBuilder.java:781)
    at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:394)
    at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:171)
    at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:340)
    at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:297)
    at org.apache.maven.project.artifact.MavenMetadataSource.retrieveRelocatedProject(MavenMetadataSource.java:577)
    at org.apache.maven.project.artifact.MavenMetadataSource.retrieve(MavenMetadataSource.java:190)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:551)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.recurse(DefaultLegacyArtifactCollector.java:605)
    at org.apache.maven.repository.legacy.resolver.DefaultLegacyArtifactCollector.collect(DefaultLegacyArtifactCollector.java:148)
    at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:504)
    at org.apache.maven.repository.legacy.LegacyRepositorySystem.resolve(LegacyRepositorySystem.java:367)
    at org.apache.maven.plugins.assembly.artifact.DefaultDependencyResolver.resolveTransitively(DefaultDependencyResolver.java:233)
    at org.apache.maven.plugins.assembly.artifact.DefaultDependencyResolver.resolve(DefaultDependencyResolver.java:119)
    at org.apache.maven.plugins.assembly.artifact.DefaultDependencyResolver.resolveDependencySets(DefaultDependencyResolver.java:155)
    at org.apache.maven.plugins.assembly.archive.phase.DependencySetAssemblyPhase.execute(DependencySetAssemblyPhase.java:95)
    at org.apache.maven.plugins.assembly.archive.DefaultAssemblyArchiver.createArchive(DefaultAssemblyArchiver.java:181)
    at io.fabric8.maven.docker.assembly.DockerAssemblyManager.createAssemblyArchive(DockerAssemblyManager.java:466)
    at io.fabric8.maven.docker.assembly.DockerAssemblyManager.createDockerTarArchive(DockerAssemblyManager.java:115)
    at io.fabric8.maven.docker.service.ArchiveService.createArchive(ArchiveService.java:123)
    at io.fabric8.maven.docker.service.ArchiveService.createArchive(ArchiveService.java:118)
    at io.fabric8.maven.docker.service.BuildService.buildImage(BuildService.java:132)
    at io.fabric8.maven.docker.service.BuildService.buildImage(BuildService.java:70)
    at io.fabric8.maven.docker.BuildMojo.buildAndTag(BuildMojo.java:74)
    at io.fabric8.maven.docker.BuildMojo.processImageConfig(BuildMojo.java:105)
    at io.fabric8.maven.docker.BuildMojo.executeInternal(BuildMojo.java:61)
    at io.fabric8.maven.docker.AbstractDockerMojo.execute(AbstractDockerMojo.java:230)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:202)
    at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:198)
    at java.util.concurrent.FutureTask.run(java.base@11.0.4/FutureTask.java:264)
    at java.util.concurrent.Executors$RunnableAdapter.call(java.base@11.0.4/Executors.java:515)
    at java.util.concurrent.FutureTask.run(java.base@11.0.4/FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.4/ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.4/ThreadPoolExecutor.java:628)
    at java.lang.Thread.run(java.base@11.0.4/Thread.java:834)