search

spotify / docker-maven-plugin

INACTIVE: A maven plugin for Docker
Apache License 2.0
2.67k stars 575 forks source link

Image creation fails if one directory is not accessible #376

Closed straumat closed 6 years ago

straumat commented 6 years ago

Description

In my project, I have one folder owned by the root user. When i try to run mvn package I get an java.nio.file.AccessDeniedException that makes the build failed.

How to reproduce

In my project, i have a Docker file and a docker-compose.yml file.

First case, If i run mvn package, all works fine : 

[INFO] Detected build of image with id a0f66ce2eba0
[INFO] Successfully built straumat/blockchain2graph:2.3-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

If I make a test by running docker-compose up, it creates a local data folder named data with different user rights : 

-rw-rw-r-- 1 straumat straumat 113990314 avril  2 16:50 blockchain2graph.jar
drwxr-xr-x 4 root     root          4096 avril  2 16:57 data
-rw-rw-r-- 1 straumat straumat      1323 avril  2 16:09 docker-compose.yml
-rw-rw-r-- 1 straumat straumat       358 avril  2 12:01 Dockerfile
-rw-rw-r-- 1 straumat straumat        38 avril  2 16:54 .dockerignore
-rw-rw-r-- 1 straumat straumat       574 avril  2 16:50 docker.iml
-rw-rw-r-- 1 straumat straumat      2980 avril  2 16:47 pom.xml
drwxrwxr-x 5 straumat straumat      4096 avril  2 16:51 target

Then, when I run mvn package, i get this exeption : 

[WARNING] An attempt failed, will retry 1 more times
org.apache.maven.plugin.MojoExecutionException: Could not build image
    at com.spotify.plugin.dockerfile.BuildMojo.buildImage(BuildMojo.java:185)
    at com.spotify.plugin.dockerfile.BuildMojo.execute(BuildMojo.java:105)
    at com.spotify.plugin.dockerfile.AbstractDockerMojo.tryExecute(AbstractDockerMojo.java:246)
    at com.spotify.plugin.dockerfile.AbstractDockerMojo.execute(AbstractDockerMojo.java:235)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: java.nio.file.AccessDeniedException: /opt/workspace/blockchain2graph/docker/data/bitcoind
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:427)
    at java.nio.file.Files.newDirectoryStream(Files.java:457)
    at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:300)
    at java.nio.file.FileTreeWalker.next(FileTreeWalker.java:372)
    at java.nio.file.Files.walkFileTree(Files.java:2706)
    at com.spotify.docker.client.CompressedDirectory.create(CompressedDirectory.java:115)
    at com.spotify.docker.client.DefaultDockerClient.build(DefaultDockerClient.java:1408)
    at com.spotify.docker.client.DefaultDockerClient.build(DefaultDockerClient.java:1383)
    at com.spotify.plugin.dockerfile.BuildMojo.buildImage(BuildMojo.java:178)
    ... 25 more
[INFO] Building Docker context /opt/workspace/blockchain2graph/docker
[INFO] 
[INFO] Image will be built as straumat/blockchain2graph:2.3-SNAPSHOT
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

I tried to add the folder in a .dockerignore but I get the same error : 

/data
/data/
/data/*
data
data/
data/*

https://github.com/straumat/blockchain2graph/tree/development/docker

What do you expect

I would expect a warning (except if the folder is in .dockerignore) but i would not expect the build to fail.

What happened instead

The build fail

Software:

thiagolsfortunato commented 6 years ago

Try: $ sudo chmod +x Dockerfile

straumat commented 6 years ago

@thiagolsfortunato It doesn't solve the issue.

thiagolsfortunato commented 6 years ago

@straumat you don't owner of the folder data, maybe the java run using straumat user.

$ sudo chown straumat:straumat -R data/

straumat commented 6 years ago

Indeed but i should not have to change it ? the plugin should not fail no ?

thiagolsfortunato commented 6 years ago

Why is created data folder when you run docker-compose.yaml ? What has inside this folder ?

mattnworb commented 6 years ago

What are the permissions on the /opt/workspace/blockchain2graph/docker/data/bitcoind file?

In general, if you run Maven as the straumat user, and a plugin like this one needs to access a file in order to compress your docker context to do work on it, then it is the OS itself that is denying the read or access of the file - and I don't see what the code running as straumat can do in that case besides report the error to you.

It would be good to verify the exact permissions on that file though (your initial report only mentions the permissions on the directory), to make sure that this is what is occurring.

straumat commented 6 years ago 
straumat@straumat-portable:/opt/workspace/blockchain2graph/docker$ sudo ls -ual /opt/workspace/blockchain2graph/docker/data/bitcoind
total 1632
drwx------ 4  999 root      4096 avril 15 12:13 .
drwxr-xr-x 4 root root      4096 avril 15 12:12 ..
-rw------- 1  999 docker      37 avril 15 12:12 banlist.dat
drwx------ 3  999 docker    4096 avril 15 12:12 blocks
drwx------ 2  999 docker    4096 avril 15 12:12 chainstate
-rw------- 1  999 docker       0 avril 15 12:12 db.log
-rw------- 1  999 docker    6164 avril 15 12:12 debug.log
-rw------- 1  999 docker  247985 avril 15 12:12 fee_estimates.dat
-rw------- 1  999 docker       0 avril 15 12:12 .lock
-rw------- 1  999 docker      17 avril 15 12:12 mempool.dat
-rw------- 1  999 docker   13344 avril 15 12:12 peers.dat
-rw------- 1  999 docker 1372160 avril 15 12:12 wallet.dat

mvn is run with straumat, docker-compose up is run with straumat but docker runs as root.

In fact, from my point of view, as the data folder is in the .dockerignore it should not not try to access it no ?

mattnworb commented 6 years ago

Can you try adding data/**/* to .dockerignore? I don't think any of the patterns mentioned earlier will match a file in a subdirectory of data, based on the rules in the documentation here.

straumat commented 6 years ago

Same error with data/**/* : 

[WARNING] An attempt failed, will retry 1 more times
org.apache.maven.plugin.MojoExecutionException: Could not build image
    at com.spotify.plugin.dockerfile.BuildMojo.buildImage(BuildMojo.java:185)
    at com.spotify.plugin.dockerfile.BuildMojo.execute(BuildMojo.java:105)
    at com.spotify.plugin.dockerfile.AbstractDockerMojo.tryExecute(AbstractDockerMojo.java:246)
    at com.spotify.plugin.dockerfile.AbstractDockerMojo.execute(AbstractDockerMojo.java:235)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: java.nio.file.AccessDeniedException: /opt/workspace/blockchain2graph/docker/data/bitcoind
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:427)
    at java.nio.file.Files.newDirectoryStream(Files.java:457)
    at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:300)
    at java.nio.file.FileTreeWalker.next(FileTreeWalker.java:372)
    at java.nio.file.Files.walkFileTree(Files.java:2706)
    at com.spotify.docker.client.CompressedDirectory.create(CompressedDirectory.java:115)
    at com.spotify.docker.client.DefaultDockerClient.build(DefaultDockerClient.java:1408)
    at com.spotify.docker.client.DefaultDockerClient.build(DefaultDockerClient.java:1383)
    at com.spotify.plugin.dockerfile.BuildMojo.buildImage(BuildMojo.java:178)
    ... 25 more
[INFO] Building Docker context /opt/workspace/blockchain2graph/docker
Scuilion commented 4 years ago

Adding an entry in .dockerignore worked for me.