search

commercialhaskell / stack

The Haskell Tool Stack
http://haskellstack.org
BSD 3-Clause "New" or "Revised" License
3.99k stars 845 forks source link

stack build shows "No directory could be located ..." inside docker container #2503

Open rainbyte opened 8 years ago

rainbyte commented 8 years ago

I'm building a project inside a docker container, with this cmd:

stack --stack-root /app/build/stack-root/ --work-dir build/work-dir/ build

stack shows this output before exit:

Populated index cache.    
Progress: 4/119No directory could be located matching the supplied path: /tmp/stack178/SHA-1.6.4.2/build

No directory could be located matching the supplied path: /tmp/stack178/ansi-terminal-0.6.2.3/build

No directory could be located matching the supplied path: /tmp/stack178/base-compat-0.9.1/build

No directory could be located matching the supplied path: /tmp/stack178/HUnit-1.3.1.1/build

After reading stack code, I found about checkOwnership function, so I tried this:

stack --allow-different-user --stack-root /app/build/stack-root/ --work-dir build/work-dir/ build

It seems to work ok (at least it started to compile dependencies).

Is this an expected behaviour?

I'm using this stack version inside haskell:7.10.3 docker image:

Version 1.1.2, Git revision cebe10e845fed4420b6224d97dcabf20477bbd4b (3646 commits) x86_64 hpack-0.14.0
borsboom commented 8 years ago

Just to be clear: are you using Stack's Docker integration, or do you just happen to be running stack in an existing container?

Does the ownership of /app/build/stack-root and build/work-dir match the current user?

rainbyte commented 8 years ago

I'm using stack inside a container. Using stack's docker support is not an option, because I was asked to integrate this service in an automated way, without having to install extra tools outside (like stack).

I've checked ownership, /app/build/stack-root and build/work-dir match current user, build is a docker volume (could it be related to this?).

Thanks in advance!

Blaisorblade commented 8 years ago

build is a docker volume (could it be related to this?).

When checking ownership, have you compared usernames or numeric user IDs? Because the latter counts. And the username <-> userid mapping on host and container are often independent. Also, can you check IDs for build itself?

rainbyte commented 8 years ago

I've compared ids and found that /app/build/stack-root and build/work-dir match current user, but build has id 0 inside the container (maybe bacause of being a volume).

I've tried using the project's folder as volume ($PWD:/app instead of $PWD:/app/build) and it worked initially. I'm using docker with these options, so user and group ids are equal between container and host

docker run --rm \
  --volume="$PWD:/app" \
  -u "$(id -u):$(id -g)" \
  sgc/services-hs-amqp-build

Next time I've tried, it failed again, this msg appeared:

Running /usr/bin/git clone --recursive https://github.com/rainbyte/openwhisk-wrapper /app/build/work-dir/downloaded/5mn7me4SBwXj in directory /app/build/work-dir/downloaded/ exited with ExitFailure 128

Cloning into '/app/build/work-dir/downloaded/5mn7me4SBwXj'...
fatal: unable to look up current user in the passwd file: no such user

https://github.com/rainbyte/openwhisk-wrapper is a package extra-dep referenced inside stack.yaml

I know that stack internal support for docker is preferred, but maybe some support for this kind of setup (running stack inside container) is useful too (at least documentation). In my case the other components of the system work inside docker (even compiled ones, using C, go, etc), without having to install almost anything (except git, docker and docker-compose)

Blaisorblade commented 8 years ago

Scratch this, I posted too early done.

rainbyte commented 8 years ago

It seems that docker works with root by default. If I leave it as is, all stack produced files need root access at host, and stack fails when not using --allow-different-user.

That's why I'm using option -u "$(id -u):$(id -g)", so it uses my user id and group. But that option doesn't update /etc/passwd (my user it is not there).

How are these problems managed when using stack internal docker support?

Blaisorblade commented 8 years ago

Three notes:

It seems that docker works with root by default.

That's defined by the Dockerfile for the image you use, or the one you build on top—see the USER command.

But that option doesn't update /etc/passwd (my user it is not there).

Correct—I fear you'd need to do it yourself.

How are these problems managed when using stack internal docker support?

No clue myself :-|. FWIW, all my Dockerfiles pick a fixed user. Even when they share files with the host (which is rather fragile, since now you need a specific config on the host :-|).

I've compared ids and found that /app/build/stack-root and build/work-dir match current user, but build has id 0 inside the container (maybe bacause of being a volume).

Yep! That's why I asked (though I didn't guess id 0). I looked at the code one week ago and didn't really get it, but IIRC some of the code paths look at the owner of the parent of build/work-dir — in particular if build/work-dir didn't exist and had to be created. Since you got a different behavior the next run, this might be what is indeed happening.

But I suspect --allow-different-user is an acceptable workaround there.

OTOH, I don't get where build comes from — your Docker cmdline mounts /app/.

Since stuff worked before (you said), maybe you added https://github.com/rainbyte/openwhisk-wrapper later or it hadn't gotten that far?

rainbyte commented 8 years ago

I'm not using USER cmd (didn't know about it), and the other components do not use it either.

Now I've tried mounting /etc/passwd as volume...

When --allow-different-user was not being used, I've received this msg:

services-hs-0.1.0.0: unregistering (missing dependencies: openwhisk-wrapper)
bytestring-builder-0.10.6.0.0: download
byteorder-1.0.4: download
auto-update-0.1.4: download
appar-0.1.4: download
No directory could be located matching the supplied path: /tmp/stack7/appar-0.1.4/build

No directory could be located matching the supplied path: /tmp/stack7/auto-update-0.1.4/build

No directory could be located matching the supplied path: /tmp/stack7/byteorder-1.0.4/build

No directory could be located matching the supplied path: /tmp/stack7/bytestring-builder-0.10.6.0.0/build

With --allow-different-user enabled, it started to compile (did not finish yet).

I've removed --allow-different-user after using option -u "$(id -u):$(id -g)" Maybe it is still needed, even if container and host users have the same uid?

The package openwhisk-wrapper was added later, as a dep for a new executable.

rainbyte commented 8 years ago

My project dir structure is something like this

build is inside services-hs, which is $PWD at host and /app inside container.

rainbyte commented 8 years ago

Ok, this time it worked!

It seems that stack inside a container needs:

And wish that other problem do not appear...

Blaisorblade commented 8 years ago

That makes sense. I'd say most of that is standard—except that usually you can take it for granted, while with Docker you have to build it on your own. That's one downside with mounting your folders inside docker rather than checking files out again :-| (though of course the latter is slower).

rainbyte commented 8 years ago

Well, I've tried using COPY instead of VOLUME, but all deps end up being downloaded each time.

I think VOLUME with custom work-dir and stack-root is the only way to keep them across builds.

borsboom commented 8 years ago

I'm a getting worried that this issue has become general Docker support rather than anything really Stack-specific. Stack works normally when run inside a Docker container but file ownership with volume mounts always complicates things (this is a general problem with Docker). Stack's built-in Docker integration works around most of these problems by doing many of the things you're doing "manually" like running as the same UID/GID, ensuring the user exists in /etc/passwd, etc.

rainbyte commented 8 years ago

@borsboom,

If using stack inside a container is not supported or not the recommended way, I think it would be better to add that info to the documentation and warn about the error msgs.

Otherwise, this use case, with all the docker "tricks" which are needed, should be documented as well. I don't think stack works normally, because all of this msgs do not appear when using locally.

I've tried to find tutorials, or some kind of docs about this, without success. All info I've found about Haskell with docker mentions:

Maybe the need for documentation about this should be inside other issue? If that the case, this issue could be closed.

borsboom commented 8 years ago

I think there are legitimate problems you identified:

Running Stack within a Docker container is totally supported, because running Stack in a Docker container is no different than running it on a bare metal machine or a VM from its point of view (when not using its built-in Docker integration, that is). However, you were effectively trying to use Stack on a misconfigured machine, which happened to be misconfigured using Docker. It's not so different than if you had a bare metal machine that mounted remote NFS filesystems with incorrect ownership are running as a user without an entry in /etc/passwd.

rainbyte commented 8 years ago

Thanks for answering, I really apreciate it.

I understand now your point about container being misconfigured, it is not a stack-specific problem. The need of VOLUME and related tricks is what affects/complicates stack use case. This could be solved with documentation and better error msgs, I suppose. If I write some kind of documentation, it could be added to stack docs?

It just surprised me (and my coworkers), because other dockerized components do not need any special config, and I couldn't explain them the issue (I want to continue using Haskell obviously).

About --allow-different-user, yes... Something weird is happening with it, becaused is needed even after solving the uid & gid thing (users are the same).

Blaisorblade commented 8 years ago

About --allow-different-user, yes... Something weird is happening with it, becaused is needed even after solving the uid & gid thing (users are the same).

As you mentioned, a directory owned by root was involved. However, I failed to reverse-engineer the semantics of the security checks that --allow-different-user disables.

rainbyte commented 8 years ago

Option --allow-different-user is needed even when build dir is owned by user 1000 (same uid at host).

When using -u "$(id -u):$(id -g)", docker handles permission correctly, no more root owned dir.

rainbyte commented 8 years ago

I've tried another approach...

If I use:

stack doesn't even reach compile stage, it gives me this:

Preventing creation of stack root '/app/build/stack-root/'. Parent directory '/app/build/' is owned by someone else.

Ownership should not matter when everyone has read and write access to build dir.

tbenst commented 7 years ago

Having similar issues over here. I tried installing packages using the location/git/commit syntax in stack.yml, but they are unable to be imported as modules. No volumes are being used.

resolver: lts-6.3

# Local packages, usually specified by relative directory name
packages:
- '.'
- location:
    git: https://github.com/mikeizbicki/subhask.git
    commit: f53fd8f465747681c88276c7dabe3646fbdf7d50
- location:
    git: https://github.com/mikeizbicki/HLearn.git
    commit: c6c0f5e7593be3867f20c56c79b998e7d7ded9fc

In Main.hs: import SubHask. After callingstack build:

 /src/app/Main.hs:12:8:
        Could not find module ‘SubHask’
        Use -v to see a list of the files searched for.

I'm also a Haskell newbie so apologies if i'm missing something simple

Blaisorblade commented 7 years ago

Is this Docker-specific? Which Stack version? This looks superficially similar to the issue in the last messages in #1176, but please don't quote me on that without more data (it might be just a coincidence). -- Paolo Giarrusso From smartphone, sorry for typos or excessive brevity

Il 30 dic 2016 17:54, "tbenst" notifications@github.com ha scritto:

Having similar issues over here. I tried installing packages using the location/git/commit syntax in stack.yml, but they are unable to be imported as modules. No volumes are being used.

resolver: lts-6.3

Local packages, usually specified by relative directory name

packages:

In Main.hs: import SubHask. After callingstack build:

/src/app/Main.hs:12:8: Could not find module ‘SubHask’ Use -v to see a list of the files searched for.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/commercialhaskell/stack/issues/2503#issuecomment-269794272, or mute the thread https://github.com/notifications/unsubscribe-auth/AARsqC3MeJ_UIbEphWeo5jlWnBpiJn2wks5rNTdGgaJpZM4JlzJr .

tbenst commented 7 years ago

Yes, here's the Dockerfile:

FROM haskell:7.10.3

WORKDIR /tmp

RUN apt-get update && apt-get install -y \
    wget \
    cmake \
    python \
    libblas-dev \
    liblapack-dev \
    git
RUN wget http://llvm.org/releases/3.5.2/llvm-3.5.2.src.tar.xz
RUN tar -xf llvm-3.5.2.src.tar.xz

WORKDIR llvm-3.5.2.src
RUN mkdir build
WORKDIR build
RUN cmake ..
RUN make -j5
RUN make install

WORKDIR /src

# this works
# RUN git clone https://github.com/mikeizbicki/subhask
# WORKDIR subhask
# RUN stack test --bench

COPY my-project /src
WORKDIR /src/my-project

# this doesn't work
RUN stack build

Edit: Version 1.1.2, Git revision cebe10e845fed4420b6224d97dcabf20477bbd4b (3646 commits) x86_64 hpack-0.14.0

Edit 2: My problem occurs outside of docker too, so I think this may be user error. Seems that i need to add packages to .cabal in addition to stack.yml

KiaraGrouwstra commented 4 years ago

I'm experiencing @rainbyte's last issue as well when attempting to Docker mount my application directory.

$ docker run -u $(id -u):$(id -g) -v $PWD:/app -ti synthesis-mount stack build
Preventing creation of stack root '/.stack/'. Parent directory '/' is owned by someone else.

Edit: user sqzlh on #nixos faced a similar issue, and solved it by setting Stack flag --work-dir to a to a subfolder to use during the build. I'm sorta confused on that tho, and just putting say stack --work-dir ./foo build there doesn't really seem to cut it.

adiykehler commented 3 years ago

I managed to reproduce the problem outside of docker. I encountered it in docker first, because there were not caches present there. But after some investigation and deleting caches (~/.cabal, ~/.stack/, build/stack-work), I also encountered it on my local system. I think the problem occurs somewhere in cabal when setting a --work-dir with stack. Removing the --work-dir parameter or setting --allow-different-user resolves the issue.

After it was successfully built once I can build to project without any problems even without the --allow-different-user parameter. That's why the bug wasn't discovered earlier for me. 

mkdir build
stack build --work-dir build/stack-work
Preparing to install GHC (tinfo6) to an isolated location.
This will not interfere with any system-level installation.
Preparing to download ghc-tinfo6-8.10.6 ...
ghc-tinfo6-8.10.6: download has begun
ghc-tinfo6-8.10.6:   11.33 MiB / 207.63 MiB (  5.46%) downloaded...
[...]
ghc-tinfo6-8.10.6:  207.63 MiB / 207.63 MiB (100.00%) downloaded...
Downloaded ghc-tinfo6-8.10.6.
Unpacking GHC into /home/USER/.stack/programs/x86_64-linux/ghc-tinfo6-8.10.6.temp/ ...
Configuring GHC ...
Installing GHC ...
Installed GHC.
Cabal file info not found for scientific-0.3.7.0@sha256:0f188a7b92780d81a2e3cf1195a3a24cfe3e7c43d0e9e0f2101a465803d68076,4773, updating
Selected mirror https://hackage.haskell.org/
Downloading root
Waiting to acquire cache lock on /home/USER/.stack/pantry/hackage/hackage-security-lock
Acquired cache lock on /home/USER/.stack/pantry/hackage/hackage-security-lock
Released cache lock on /home/USER/.stack/pantry/hackage/hackage-security-lock
Selected mirror https://hackage.haskell.org/
Downloading timestamp
Downloading snapshot
Downloading mirrors
Cannot update index (no local copy)
Downloading index
Waiting to acquire cache lock on /home/USER/.stack/pantry/hackage/hackage-security-lock
Acquired cache lock on /home/USER/.stack/pantry/hackage/hackage-security-lock
Released cache lock on /home/USER/.stack/pantry/hackage/hackage-security-lock
Updated package index downloaded
Calculating hashes to check for hackage-security rebases or filesystem changes
No old cache found, populating cache from scratch
Populating cache from file size 747617792, hash b0348cca768ad3ca54d4e72fa8279a0ac53bf42eee574d8f1ed253fccdf80b66
Populating package index cache ...
Processed 400 cabal files
[...]
Processed 151200 cabal files
Package index cache populated
[1 of 2] Compiling Main             ( /home/USER/.stack/setup-exe-src/setup-mPHDZzAJ.hs, /home/USER/.stack/setup-exe-src/setup-mPHDZzAJ.o )
[2 of 2] Compiling StackSetupShim   ( /home/USER/.stack/setup-exe-src/setup-shim-mPHDZzAJ.hs, /home/USER/.stack/setup-exe-src/setup-shim-mPHDZzAJ.o )
Linking /home/USER/.stack/setup-exe-cache/x86_64-linux-tinfo6/tmp-Cabal-simple_mPHDZzAJ_3.2.1.0_ghc-8.10.6 ...
kotlin-connectors-docu-generation-0.1.0.0: unregistering
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/basement-0.0.12/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/blaze-builder-0.4.2.1/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/base64-bytestring-1.1.0.0/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/base16-bytestring-1.0.1.0/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/base-orphans-0.8.4/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/base-compat-0.11.2/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/SHA-1.6.4.4/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/HsYAML-0.2.1.0/build