search

fabric8io / docker-maven-plugin

Maven plugin for running and creating Docker images
https://dmp.fabric8.io
Apache License 2.0
1.88k stars 643 forks source link

docker:stop fails to stop and remove container on multi-module projects #1517

Open j-be opened 2 years ago

j-be commented 2 years ago

Description

I'm having the weirdest of errors: After I run my maven builds a couple of times (including failures, which do not clean up the containers) the plugin suddenly fails to stop the container.

mvn docker:stop -X gives:

[INFO] --- docker-maven-plugin:0.38.1:stop (default-cli) @ util-cache-redisson ---
[DEBUG] Using mirror maven-default-http-blocker (http://0.0.0.0/) for ow2-snapshot (http://repository.ow2.org/nexus/content/repositories/snapshots).
[DEBUG] Using mirror maven-default-http-blocker (http://0.0.0.0/) for codehaus.snapshots (http://snapshots.repository.codehaus.org).
[DEBUG] Using mirror maven-default-http-blocker (http://0.0.0.0/) for codehaus-snapshots (http://nexus.codehaus.org/snapshots/).
[DEBUG] Dependency collection stats {ConflictMarker.analyzeTime=976942, ConflictMarker.markTime=408192, ConflictMarker.nodeCount=248, ConflictIdSorter.graphTime=554572, ConflictIdSorter.topsortTime=266059, ConflictIdSorter.conflictIdCount=91, ConflictIdSorter.conflictIdCycleCount=0, ConflictResolver.totalTime=6244922, ConflictResolver.conflictItemCount=208, DefaultDependencyCollector.collectTime=258138646, DefaultDependencyCollector.transformTime=9795874}
[DEBUG] io.fabric8:docker-maven-plugin:jar:0.38.1
[DEBUG]    org.apache.httpcomponents:httpclient:jar:4.5.13:compile
[DEBUG]       org.apache.httpcomponents:httpcore:jar:4.4.13:compile
[DEBUG]       commons-logging:commons-logging:jar:1.2:compile
[DEBUG]       commons-codec:commons-codec:jar:1.11:compile
[DEBUG]    com.github.jnr:jnr-unixsocket:jar:0.38.14:compile
[DEBUG]       com.github.jnr:jnr-ffi:jar:2.2.10:compile
[DEBUG]          com.github.jnr:jffi:jar:1.3.8:compile
[DEBUG]          com.github.jnr:jffi:jar:native:1.3.8:runtime
[DEBUG]          org.ow2.asm:asm-commons:jar:9.2:compile
[DEBUG]          org.ow2.asm:asm-analysis:jar:9.2:compile
[DEBUG]          org.ow2.asm:asm-tree:jar:9.2:compile
[DEBUG]          org.ow2.asm:asm-util:jar:9.2:compile
[DEBUG]          com.github.jnr:jnr-a64asm:jar:1.0.0:compile
[DEBUG]          com.github.jnr:jnr-x86asm:jar:1.0.2:compile
[DEBUG]       com.github.jnr:jnr-constants:jar:0.10.3:compile
[DEBUG]       com.github.jnr:jnr-enxio:jar:0.32.12:compile
[DEBUG]       com.github.jnr:jnr-posix:jar:3.1.13:compile
[DEBUG]    org.apache.commons:commons-lang3:jar:3.6:compile
[DEBUG]    org.apache.commons:commons-text:jar:1.1:compile
[DEBUG]    com.google.code.gson:gson:jar:2.8.5:compile
[DEBUG]    org.fusesource.jansi:jansi:jar:1.16:compile
[DEBUG]    org.bouncycastle:bcpkix-jdk15on:jar:1.65:compile
[DEBUG]       org.bouncycastle:bcprov-jdk15on:jar:1.65:compile
[DEBUG]    org.apache.maven.plugins:maven-assembly-plugin:jar:3.1.0:compile
[DEBUG]       org.apache.maven:maven-plugin-api:jar:3.0:compile
[DEBUG]          org.sonatype.sisu:sisu-inject-plexus:jar:1.4.2:compile
[DEBUG]             org.sonatype.sisu:sisu-inject-bean:jar:1.4.2:compile
[DEBUG]                org.sonatype.sisu:sisu-guice:jar:noaop:2.1.7:compile
[DEBUG]       org.apache.maven:maven-core:jar:3.0:compile
[DEBUG]          org.apache.maven:maven-settings:jar:3.0:compile
[DEBUG]          org.apache.maven:maven-settings-builder:jar:3.0:compile
[DEBUG]          org.apache.maven:maven-repository-metadata:jar:3.0:compile
[DEBUG]          org.apache.maven:maven-model-builder:jar:3.0:compile
[DEBUG]          org.apache.maven:maven-aether-provider:jar:3.0:runtime
[DEBUG]          org.sonatype.aether:aether-impl:jar:1.7:compile
[DEBUG]             org.sonatype.aether:aether-spi:jar:1.7:compile
[DEBUG]          org.sonatype.aether:aether-api:jar:1.7:compile
[DEBUG]          org.sonatype.aether:aether-util:jar:1.7:compile
[DEBUG]          org.codehaus.plexus:plexus-classworlds:jar:2.2.3:compile
[DEBUG]          org.codehaus.plexus:plexus-component-annotations:jar:1.5.5:compile
[DEBUG]          org.sonatype.plexus:plexus-sec-dispatcher:jar:1.3:compile
[DEBUG]             org.sonatype.plexus:plexus-cipher:jar:1.4:compile
[DEBUG]       org.apache.maven:maven-artifact:jar:3.0:compile
[DEBUG]       org.apache.maven:maven-model:jar:3.0:compile
[DEBUG]       org.apache.maven.shared:maven-common-artifact-filters:jar:3.0.1:compile
[DEBUG]          org.apache.maven.shared:maven-shared-utils:jar:3.1.0:compile
[DEBUG]       org.apache.maven.shared:maven-artifact-transfer:jar:0.9.0:compile
[DEBUG]          org.slf4j:slf4j-api:jar:1.7.5:compile
[DEBUG]       org.codehaus.plexus:plexus-archiver:jar:3.5:compile
[DEBUG]          org.iq80.snappy:snappy:jar:0.4:compile
[DEBUG]          org.tukaani:xz:jar:1.6:runtime
[DEBUG]       org.apache.maven.shared:file-management:jar:3.0.0:compile
[DEBUG]       org.apache.maven.shared:maven-shared-io:jar:3.0.0:compile
[DEBUG]          org.apache.maven:maven-compat:jar:3.0:compile
[DEBUG]          org.apache.maven.wagon:wagon-provider-api:jar:2.10:compile
[DEBUG]       commons-io:commons-io:jar:2.5:compile
[DEBUG]       org.apache.maven.shared:maven-filtering:jar:3.1.1:compile
[DEBUG]          org.sonatype.plexus:plexus-build-api:jar:0.0.7:compile
[DEBUG]       org.codehaus.plexus:plexus-io:jar:3.0.0:compile
[DEBUG]       org.apache.maven:maven-archiver:jar:3.2.0:compile
[DEBUG]       org.codehaus.plexus:plexus-utils:jar:3.1.0:compile
[DEBUG]    com.google.cloud.tools:jib-core:jar:0.18.0:compile
[DEBUG]       com.google.cloud.tools:jib-build-plan:jar:0.4.0:compile
[DEBUG]       com.google.http-client:google-http-client:jar:1.38.1:runtime
[DEBUG]          io.opencensus:opencensus-api:jar:0.24.0:runtime
[DEBUG]             io.grpc:grpc-context:jar:1.22.1:runtime
[DEBUG]          io.opencensus:opencensus-contrib-http-util:jar:0.24.0:runtime
[DEBUG]       com.google.http-client:google-http-client-apache-v2:jar:1.38.1:runtime
[DEBUG]       com.google.auth:google-auth-library-oauth2-http:jar:0.22.2:runtime
[DEBUG]          com.google.auto.value:auto-value-annotations:jar:1.7.4:runtime
[DEBUG]          com.google.auth:google-auth-library-credentials:jar:0.22.2:runtime
[DEBUG]          com.google.http-client:google-http-client-jackson2:jar:1.38.0:runtime
[DEBUG]       org.apache.commons:commons-compress:jar:1.20:compile
[DEBUG]       com.fasterxml.jackson.core:jackson-databind:jar:2.12.1:runtime
[DEBUG]          com.fasterxml.jackson.core:jackson-annotations:jar:2.12.1:runtime
[DEBUG]          com.fasterxml.jackson.core:jackson-core:jar:2.12.1:runtime
[DEBUG]       com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.12.1:runtime
[DEBUG]       org.ow2.asm:asm:jar:9.1:compile
[DEBUG]    org.yaml:snakeyaml:jar:1.26:compile
[DEBUG]    com.google.guava:guava:jar:30.1-jre:compile
[DEBUG]       com.google.guava:failureaccess:jar:1.0.1:compile
[DEBUG]       com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
[DEBUG]       com.google.code.findbugs:jsr305:jar:3.0.2:compile
[DEBUG]       org.checkerframework:checker-qual:jar:3.5.0:compile
[DEBUG]       com.google.errorprone:error_prone_annotations:jar:2.3.4:compile
[DEBUG]       com.google.j2objc:j2objc-annotations:jar:1.3:compile
[DEBUG]    org.codehaus.plexus:plexus-interpolation:jar:1.24:compile
[DEBUG]    net.jodah:failsafe:jar:2.4.0:compile
[DEBUG] Created new class realm plugin>io.fabric8:docker-maven-plugin:0.38.1
[DEBUG] Importing foreign packages into class realm plugin>io.fabric8:docker-maven-plugin:0.38.1
[DEBUG]   Imported:  < maven.api
[DEBUG] Populating class realm plugin>io.fabric8:docker-maven-plugin:0.38.1
[DEBUG]   Included: io.fabric8:docker-maven-plugin:jar:0.38.1
[DEBUG]   Included: org.apache.httpcomponents:httpclient:jar:4.5.13
[DEBUG]   Included: org.apache.httpcomponents:httpcore:jar:4.4.13
[DEBUG]   Included: commons-logging:commons-logging:jar:1.2
[DEBUG]   Included: commons-codec:commons-codec:jar:1.11
[DEBUG]   Included: com.github.jnr:jnr-unixsocket:jar:0.38.14
[DEBUG]   Included: com.github.jnr:jnr-ffi:jar:2.2.10
[DEBUG]   Included: com.github.jnr:jffi:jar:1.3.8
[DEBUG]   Included: com.github.jnr:jffi:jar:native:1.3.8
[DEBUG]   Included: org.ow2.asm:asm-commons:jar:9.2
[DEBUG]   Included: org.ow2.asm:asm-analysis:jar:9.2
[DEBUG]   Included: org.ow2.asm:asm-tree:jar:9.2
[DEBUG]   Included: org.ow2.asm:asm-util:jar:9.2
[DEBUG]   Included: com.github.jnr:jnr-a64asm:jar:1.0.0
[DEBUG]   Included: com.github.jnr:jnr-x86asm:jar:1.0.2
[DEBUG]   Included: com.github.jnr:jnr-constants:jar:0.10.3
[DEBUG]   Included: com.github.jnr:jnr-enxio:jar:0.32.12
[DEBUG]   Included: com.github.jnr:jnr-posix:jar:3.1.13
[DEBUG]   Included: org.apache.commons:commons-lang3:jar:3.6
[DEBUG]   Included: org.apache.commons:commons-text:jar:1.1
[DEBUG]   Included: com.google.code.gson:gson:jar:2.8.5
[DEBUG]   Included: org.bouncycastle:bcpkix-jdk15on:jar:1.65
[DEBUG]   Included: org.bouncycastle:bcprov-jdk15on:jar:1.65
[DEBUG]   Included: org.apache.maven.plugins:maven-assembly-plugin:jar:3.1.0
[DEBUG]   Included: org.sonatype.sisu:sisu-inject-bean:jar:1.4.2
[DEBUG]   Included: org.sonatype.sisu:sisu-guice:jar:noaop:2.1.7
[DEBUG]   Included: org.sonatype.aether:aether-util:jar:1.7
[DEBUG]   Included: org.codehaus.plexus:plexus-component-annotations:jar:1.5.5
[DEBUG]   Included: org.sonatype.plexus:plexus-sec-dispatcher:jar:1.3
[DEBUG]   Included: org.sonatype.plexus:plexus-cipher:jar:1.4
[DEBUG]   Included: org.apache.maven.shared:maven-common-artifact-filters:jar:3.0.1
[DEBUG]   Included: org.apache.maven.shared:maven-shared-utils:jar:3.1.0
[DEBUG]   Included: org.apache.maven.shared:maven-artifact-transfer:jar:0.9.0
[DEBUG]   Included: org.codehaus.plexus:plexus-archiver:jar:3.5
[DEBUG]   Included: org.iq80.snappy:snappy:jar:0.4
[DEBUG]   Included: org.tukaani:xz:jar:1.6
[DEBUG]   Included: org.apache.maven.shared:file-management:jar:3.0.0
[DEBUG]   Included: org.apache.maven.shared:maven-shared-io:jar:3.0.0
[DEBUG]   Included: commons-io:commons-io:jar:2.5
[DEBUG]   Included: org.apache.maven.shared:maven-filtering:jar:3.1.1
[DEBUG]   Included: org.sonatype.plexus:plexus-build-api:jar:0.0.7
[DEBUG]   Included: org.codehaus.plexus:plexus-io:jar:3.0.0
[DEBUG]   Included: org.apache.maven:maven-archiver:jar:3.2.0
[DEBUG]   Included: org.codehaus.plexus:plexus-utils:jar:3.1.0
[DEBUG]   Included: com.google.cloud.tools:jib-core:jar:0.18.0
[DEBUG]   Included: com.google.cloud.tools:jib-build-plan:jar:0.4.0
[DEBUG]   Included: com.google.http-client:google-http-client:jar:1.38.1
[DEBUG]   Included: io.opencensus:opencensus-api:jar:0.24.0
[DEBUG]   Included: io.grpc:grpc-context:jar:1.22.1
[DEBUG]   Included: io.opencensus:opencensus-contrib-http-util:jar:0.24.0
[DEBUG]   Included: com.google.http-client:google-http-client-apache-v2:jar:1.38.1
[DEBUG]   Included: com.google.auth:google-auth-library-oauth2-http:jar:0.22.2
[DEBUG]   Included: com.google.auto.value:auto-value-annotations:jar:1.7.4
[DEBUG]   Included: com.google.auth:google-auth-library-credentials:jar:0.22.2
[DEBUG]   Included: com.google.http-client:google-http-client-jackson2:jar:1.38.0
[DEBUG]   Included: org.apache.commons:commons-compress:jar:1.20
[DEBUG]   Included: com.fasterxml.jackson.core:jackson-databind:jar:2.12.1
[DEBUG]   Included: com.fasterxml.jackson.core:jackson-annotations:jar:2.12.1
[DEBUG]   Included: com.fasterxml.jackson.core:jackson-core:jar:2.12.1
[DEBUG]   Included: com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.12.1
[DEBUG]   Included: org.ow2.asm:asm:jar:9.1
[DEBUG]   Included: org.yaml:snakeyaml:jar:1.26
[DEBUG]   Included: com.google.guava:guava:jar:30.1-jre
[DEBUG]   Included: com.google.guava:failureaccess:jar:1.0.1
[DEBUG]   Included: com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava
[DEBUG]   Included: com.google.code.findbugs:jsr305:jar:3.0.2
[DEBUG]   Included: org.checkerframework:checker-qual:jar:3.5.0
[DEBUG]   Included: com.google.errorprone:error_prone_annotations:jar:2.3.4
[DEBUG]   Included: com.google.j2objc:j2objc-annotations:jar:1.3
[DEBUG]   Included: org.codehaus.plexus:plexus-interpolation:jar:1.24
[DEBUG]   Included: net.jodah:failsafe:jar:2.4.0
[DEBUG] Configuring mojo io.fabric8:docker-maven-plugin:0.38.1:stop from plugin realm ClassRealm[plugin>io.fabric8:docker-maven-plugin:0.38.1, parent: jdk.internal.loader.ClassLoaders$AppClassLoader@277050dc]
[DEBUG] Configuring mojo 'io.fabric8:docker-maven-plugin:0.38.1:stop' with basic configurator -->
[DEBUG]   (f) allContainers = false
[DEBUG]   (f) autoCreateCustomNetworks = false
[DEBUG]   (f) executeStopOnVMShutdown = false
[DEBUG]   (f) execution = io.fabric8:docker-maven-plugin:0.38.1:stop {execution: default-cli}
[DEBUG]   (s) name = jtiss-test-redis
[DEBUG]   (f) stopNamePattern = jtiss-test-redis-*
[DEBUG]   (f) from = redis
[DEBUG]   (f) build = io.fabric8.maven.docker.config.BuildImageConfiguration@64db4967
[DEBUG]   (f) ports = [6379:6379]
[DEBUG]   (f) host = localhost
[DEBUG]   (f) ports = [6379]
[DEBUG]   (f) tcp = io.fabric8.maven.docker.config.WaitConfiguration$TcpConfiguration@37b52340
[DEBUG]   (f) time = 10000
[DEBUG]   (f) wait = io.fabric8.maven.docker.config.WaitConfiguration@663bb8ef
[DEBUG]   (f) run = io.fabric8.maven.docker.config.RunImageConfiguration@2f4e40d7
[DEBUG]   (f) images = [ImageConfiguration {name='jtiss-test-redis', alias='null'}]
[DEBUG]   (f) jib = false
[DEBUG]   (f) jibImageFormat = docker
[DEBUG]   (f) keepContainer = false
[DEBUG]   (f) keepRunning = false
[DEBUG]   (f) logStdout = false
[DEBUG]   (f) maxConnections = 100
[DEBUG]   (f) outputDirectory = target/docker
[DEBUG]   (f) project = MavenProject: at.ac.tuwien.tiss.util:util-cache-redisson:2.15.1-SNAPSHOT @ /home/juri/workspace/jtiss/util/cache-redisson/pom.xml
[DEBUG]   (f) removeVolumes = false
[DEBUG]   (f) session = org.apache.maven.execution.MavenSession@7e5843db
[DEBUG]   (f) settings = org.apache.maven.execution.SettingsAdapter@459f703f
[DEBUG]   (f) skip = false
[DEBUG]   (f) skipExtendedAuth = false
[DEBUG]   (f) skipMachine = false
[DEBUG]   (f) sledgeHammer = false
[DEBUG]   (f) sourceDirectory = src/main/docker
[DEBUG]   (f) useColor = true
[DEBUG] -- end configuration --
[DEBUG] DOCKER> Converted stopNamePattern jtiss-test-redis-* into image name regular expression ^jtiss-test-redis-([^/:]|:(?=.*:))*$
[DEBUG] DOCKER> Converted stopNamePattern jtiss-test-redis-* into container name regular expression ^jtiss-test-redis-([^/:]|:(?=.*:))*$
[DEBUG] DOCKER> Container name jtiss-test-redis-1 matched stopNamePattern
[DEBUG] DOCKER> Neither container image mongo-express nor name mongodb_mongo-express_1 matched stopNamePattern
[DEBUG] DOCKER> Neither container image mongo:4.4 nor name mongodb_mongo_1 matched stopNamePattern
[DEBUG] DOCKER> Neither container image portainer/portainer-ce nor name portainer matched stopNamePattern
[DEBUG] DOCKER> Neither container image sonarqube nor name sonarqube matched stopNamePattern
[DEBUG] DOCKER> Neither container image redis nor name test-redis matched stopNamePattern
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.038 s
[INFO] Finished at: 2022-01-20T16:00:24+01:00
[INFO] ------------------------------------------------------------------------

Notice the missing:

[INFO] DOCKER> [jtiss-test-redis:latest]: Stop and removed container 71d3c8dc5faa after 0 ms

After deleting the container manually and starting over everything is back to normal. Then it works for a couple of runs, and out of nowhere it starts failing again. I did NOT yet find a way to reproduce it deterministically. See comment below.

My config:

                    <plugin>
                        <groupId>io.fabric8</groupId>
                        <artifactId>docker-maven-plugin</artifactId>
                        <configuration>
                            <images>
                                <image>
                                    <name>jtiss-test-redis</name>
                                    <stopNamePattern>jtiss-test-redis-*</stopNamePattern>
                                    <build>
                                        <from>redis</from>
                                    </build>
                                    <run>
                                        <ports>
                                            <port>6379:6379</port>
                                        </ports>
                                        <wait>
                                            <tcp>
                                                <host>localhost</host>
                                                <ports>6379</ports>
                                            </tcp>
                                            <time>10000</time>
                                        </wait>
                                    </run>
                                </image>
                            </images>
                        </configuration>
                    </plugin>

and in pluginManagement:

                <plugin>
                    <groupId>io.fabric8</groupId>
                    <artifactId>docker-maven-plugin</artifactId>
                    <version>0.38.1</version>
                    <executions>
                        <execution>
                            <id>cleanup</id>
                            <phase>test-compile</phase>
                            <goals>
                                <goal>stop</goal>
                                <goal>remove</goal>
                            </goals>
                        </execution>
                        <execution>
                            <id>start</id>
                            <phase>test-compile</phase>
                            <goals>
                                <goal>build</goal>
                                <goal>start</goal>
                            </goals>
                        </execution>
                        <execution>
                            <id>stop</id>
                            <phase>post-integration-test</phase>
                            <goals>
                                <goal>stop</goal>
                                <goal>remove</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>

Info

j-be commented 2 years ago

I found how to reproduce it:

In a multi-module project (assume you have the modules one and other):

Now the container is still there, even though the same project (just a different module) that created it is trying to stop it.

A small reproducer can be found at https://github.com/j-be/fabric8io_docker-maven-plugin_1517

j-be commented 2 years ago

Update: setting docker.allContainers to true in pom.xml, like:

<properties>
    <docker.allContainers>true</docker.allContainers>
</properties>

also solves the issue.

coolbeevip commented 2 years ago

Hi @j-be

Where is the description of docker.allContainers?

j-be commented 2 years ago

Hi @coolbeevip

I'm not sure what you mean by "description". It is documented here, see Table 20.