Open rainbyte opened 8 years ago
Just to be clear: are you using Stack's Docker integration, or do you just happen to be running stack
in an existing container?
Does the ownership of /app/build/stack-root
and build/work-dir
match the current user?
I'm using stack inside a container. Using stack's docker support is not an option, because I was asked to integrate this service in an automated way, without having to install extra tools outside (like stack).
I've checked ownership, /app/build/stack-root
and build/work-dir
match current user, build
is a docker volume (could it be related to this?).
Thanks in advance!
build is a docker volume (could it be related to this?).
When checking ownership, have you compared usernames or numeric user IDs? Because the latter counts. And the username <-> userid mapping on host and container are often independent.
Also, can you check IDs for build
itself?
I've compared ids and found that /app/build/stack-root
and build/work-dir
match current user, but build
has id 0 inside the container (maybe bacause of being a volume).
I've tried using the project's folder as volume ($PWD:/app
instead of $PWD:/app/build
) and it worked initially.
I'm using docker with these options, so user and group ids are equal between container and host
docker run --rm \
--volume="$PWD:/app" \
-u "$(id -u):$(id -g)" \
sgc/services-hs-amqp-build
Next time I've tried, it failed again, this msg appeared:
Running /usr/bin/git clone --recursive https://github.com/rainbyte/openwhisk-wrapper /app/build/work-dir/downloaded/5mn7me4SBwXj in directory /app/build/work-dir/downloaded/ exited with ExitFailure 128
Cloning into '/app/build/work-dir/downloaded/5mn7me4SBwXj'...
fatal: unable to look up current user in the passwd file: no such user
https://github.com/rainbyte/openwhisk-wrapper
is a package extra-dep referenced inside stack.yaml
I know that stack internal support for docker is preferred, but maybe some support for this kind of setup (running stack inside container) is useful too (at least documentation). In my case the other components of the system work inside docker (even compiled ones, using C, go, etc), without having to install almost anything (except git, docker and docker-compose)
Scratch this, I posted too early done.
It seems that docker works with root by default. If I leave it as is, all stack produced files need root access at host, and stack fails when not using --allow-different-user
.
That's why I'm using option -u "$(id -u):$(id -g)"
, so it uses my user id and group.
But that option doesn't update /etc/passwd (my user it is not there).
How are these problems managed when using stack internal docker support?
Three notes:
It seems that docker works with root by default.
That's defined by the Dockerfile for the image you use, or the one you build on top—see the USER
command.
But that option doesn't update /etc/passwd (my user it is not there).
Correct—I fear you'd need to do it yourself.
How are these problems managed when using stack internal docker support?
No clue myself :-|. FWIW, all my Dockerfiles pick a fixed user. Even when they share files with the host (which is rather fragile, since now you need a specific config on the host :-|).
I've compared ids and found that /app/build/stack-root and build/work-dir match current user, but build has id 0 inside the container (maybe bacause of being a volume).
Yep! That's why I asked (though I didn't guess id 0). I looked at the code one week ago and didn't really get it, but IIRC some of the code paths look at the owner of the parent of build/work-dir
— in particular if build/work-dir
didn't exist and had to be created. Since you got a different behavior the next run, this might be what is indeed happening.
But I suspect --allow-different-user
is an acceptable workaround there.
OTOH, I don't get where build
comes from — your Docker cmdline mounts /app/
.
Since stuff worked before (you said), maybe you added https://github.com/rainbyte/openwhisk-wrapper
later or it hadn't gotten that far?
I'm not using USER cmd (didn't know about it), and the other components do not use it either.
Now I've tried mounting /etc/passwd as volume...
When --allow-different-user
was not being used, I've received this msg:
services-hs-0.1.0.0: unregistering (missing dependencies: openwhisk-wrapper)
bytestring-builder-0.10.6.0.0: download
byteorder-1.0.4: download
auto-update-0.1.4: download
appar-0.1.4: download
No directory could be located matching the supplied path: /tmp/stack7/appar-0.1.4/build
No directory could be located matching the supplied path: /tmp/stack7/auto-update-0.1.4/build
No directory could be located matching the supplied path: /tmp/stack7/byteorder-1.0.4/build
No directory could be located matching the supplied path: /tmp/stack7/bytestring-builder-0.10.6.0.0/build
With --allow-different-user
enabled, it started to compile (did not finish yet).
I've removed --allow-different-user
after using option -u "$(id -u):$(id -g)"
Maybe it is still needed, even if container and host users have the same uid?
The package openwhisk-wrapper was added later, as a dep for a new executable.
My project dir structure is something like this
build
is inside services-hs
, which is $PWD
at host and /app
inside container.
Ok, this time it worked!
It seems that stack
inside a container needs:
-u "$(id -u):$(id -g)"
--allow-different-user
enabledAnd wish that other problem do not appear...
That makes sense. I'd say most of that is standard—except that usually you can take it for granted, while with Docker you have to build it on your own. That's one downside with mounting your folders inside docker rather than checking files out again :-| (though of course the latter is slower).
Well, I've tried using COPY instead of VOLUME, but all deps end up being downloaded each time.
I think VOLUME with custom work-dir and stack-root is the only way to keep them across builds.
I'm a getting worried that this issue has become general Docker support rather than anything really Stack-specific. Stack works normally when run inside a Docker container but file ownership with volume mounts always complicates things (this is a general problem with Docker). Stack's built-in Docker integration works around most of these problems by doing many of the things you're doing "manually" like running as the same UID/GID, ensuring the user exists in /etc/passwd, etc.
@borsboom,
If using stack
inside a container is not supported or not the recommended way, I think it would be better to add that info to the documentation and warn about the error msgs.
Otherwise, this use case, with all the docker "tricks" which are needed, should be documented as well.
I don't think stack works normally
, because all of this msgs do not appear when using locally.
I've tried to find tutorials, or some kind of docs about this, without success. All info I've found about Haskell with docker mentions:
docker: true
to stack.yamlMaybe the need for documentation about this should be inside other issue? If that the case, this issue could be closed.
I think there are legitimate problems you identified:
--work-dir build/work-dir/
when it comes to installing dependencies, and I have no idea why --allow-different-user
affects it.--allow-different-user
is not documented prominently enough and also in any case where it is needed, Stack should actually suggest it in order to be self-documenting. It already does that in most cases, but not here (which looks to be a strange edge case).Running Stack within a Docker container is totally supported, because running Stack in a Docker container is no different than running it on a bare metal machine or a VM from its point of view (when not using its built-in Docker integration, that is). However, you were effectively trying to use Stack on a misconfigured machine, which happened to be misconfigured using Docker. It's not so different than if you had a bare metal machine that mounted remote NFS filesystems with incorrect ownership are running as a user without an entry in /etc/passwd
.
Thanks for answering, I really apreciate it.
I understand now your point about container being misconfigured, it is not a stack-specific problem. The need of VOLUME and related tricks is what affects/complicates stack use case. This could be solved with documentation and better error msgs, I suppose. If I write some kind of documentation, it could be added to stack docs?
It just surprised me (and my coworkers), because other dockerized components do not need any special config, and I couldn't explain them the issue (I want to continue using Haskell obviously).
About --allow-different-user
, yes... Something weird is happening with it, becaused is needed even after solving the uid & gid thing (users are the same).
About --allow-different-user, yes... Something weird is happening with it, becaused is needed even after solving the uid & gid thing (users are the same).
As you mentioned, a directory owned by root was involved. However, I failed to reverse-engineer the semantics of the security checks that --allow-different-user
disables.
Option --allow-different-user
is needed even when build
dir is owned by user 1000 (same uid at host).
When using -u "$(id -u):$(id -g)"
, docker handles permission correctly, no more root owned dir.
I've tried another approach...
If I use:
--allow-different-user
enabledstack doesn't even reach compile stage, it gives me this:
Preventing creation of stack root '/app/build/stack-root/'. Parent directory '/app/build/' is owned by someone else.
Ownership should not matter when everyone has read and write access to build dir.
Having similar issues over here. I tried installing packages using the location/git/commit syntax in stack.yml, but they are unable to be imported as modules. No volumes are being used.
resolver: lts-6.3
# Local packages, usually specified by relative directory name
packages:
- '.'
- location:
git: https://github.com/mikeizbicki/subhask.git
commit: f53fd8f465747681c88276c7dabe3646fbdf7d50
- location:
git: https://github.com/mikeizbicki/HLearn.git
commit: c6c0f5e7593be3867f20c56c79b998e7d7ded9fc
In Main.hs: import SubHask
. After callingstack build
:
/src/app/Main.hs:12:8:
Could not find module ‘SubHask’
Use -v to see a list of the files searched for.
I'm also a Haskell newbie so apologies if i'm missing something simple
Is this Docker-specific? Which Stack version? This looks superficially similar to the issue in the last messages in #1176, but please don't quote me on that without more data (it might be just a coincidence). -- Paolo Giarrusso From smartphone, sorry for typos or excessive brevity
Il 30 dic 2016 17:54, "tbenst" notifications@github.com ha scritto:
Having similar issues over here. I tried installing packages using the location/git/commit syntax in stack.yml, but they are unable to be imported as modules. No volumes are being used.
resolver: lts-6.3
packages:
In Main.hs: import SubHask. After callingstack build:
/src/app/Main.hs:12:8: Could not find module ‘SubHask’ Use -v to see a list of the files searched for.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/commercialhaskell/stack/issues/2503#issuecomment-269794272, or mute the thread https://github.com/notifications/unsubscribe-auth/AARsqC3MeJ_UIbEphWeo5jlWnBpiJn2wks5rNTdGgaJpZM4JlzJr .
Yes, here's the Dockerfile:
FROM haskell:7.10.3
WORKDIR /tmp
RUN apt-get update && apt-get install -y \
wget \
cmake \
python \
libblas-dev \
liblapack-dev \
git
RUN wget http://llvm.org/releases/3.5.2/llvm-3.5.2.src.tar.xz
RUN tar -xf llvm-3.5.2.src.tar.xz
WORKDIR llvm-3.5.2.src
RUN mkdir build
WORKDIR build
RUN cmake ..
RUN make -j5
RUN make install
WORKDIR /src
# this works
# RUN git clone https://github.com/mikeizbicki/subhask
# WORKDIR subhask
# RUN stack test --bench
COPY my-project /src
WORKDIR /src/my-project
# this doesn't work
RUN stack build
Edit: Version 1.1.2, Git revision cebe10e845fed4420b6224d97dcabf20477bbd4b (3646 commits) x86_64 hpack-0.14.0
Edit 2: My problem occurs outside of docker too, so I think this may be user error. Seems that i need to add packages to .cabal in addition to stack.yml
I'm experiencing @rainbyte's last issue as well when attempting to Docker mount my application directory.
$ docker run -u $(id -u):$(id -g) -v $PWD:/app -ti synthesis-mount stack build
Preventing creation of stack root '/.stack/'. Parent directory '/' is owned by someone else.
Edit: user sqzlh
on #nixos
faced a similar issue, and solved it by setting Stack flag --work-dir
to a to a subfolder to use during the build. I'm sorta confused on that tho, and just putting say stack --work-dir ./foo build
there doesn't really seem to cut it.
I managed to reproduce the problem outside of docker. I encountered it in docker first, because there were not caches present there. But after some investigation and deleting caches (~/.cabal
, ~/.stack/
, build/stack-work
), I also encountered it on my local system. I think the problem occurs somewhere in cabal when setting a --work-dir
with stack.
Removing the --work-dir
parameter or setting --allow-different-user
resolves the issue.
After it was successfully built once I can build to project without any problems even without the --allow-different-user
parameter. That's why the bug wasn't discovered earlier for me.
mkdir build
stack build --work-dir build/stack-work
Preparing to install GHC (tinfo6) to an isolated location.
This will not interfere with any system-level installation.
Preparing to download ghc-tinfo6-8.10.6 ...
ghc-tinfo6-8.10.6: download has begun
ghc-tinfo6-8.10.6: 11.33 MiB / 207.63 MiB ( 5.46%) downloaded...
[...]
ghc-tinfo6-8.10.6: 207.63 MiB / 207.63 MiB (100.00%) downloaded...
Downloaded ghc-tinfo6-8.10.6.
Unpacking GHC into /home/USER/.stack/programs/x86_64-linux/ghc-tinfo6-8.10.6.temp/ ...
Configuring GHC ...
Installing GHC ...
Installed GHC.
Cabal file info not found for scientific-0.3.7.0@sha256:0f188a7b92780d81a2e3cf1195a3a24cfe3e7c43d0e9e0f2101a465803d68076,4773, updating
Selected mirror https://hackage.haskell.org/
Downloading root
Waiting to acquire cache lock on /home/USER/.stack/pantry/hackage/hackage-security-lock
Acquired cache lock on /home/USER/.stack/pantry/hackage/hackage-security-lock
Released cache lock on /home/USER/.stack/pantry/hackage/hackage-security-lock
Selected mirror https://hackage.haskell.org/
Downloading timestamp
Downloading snapshot
Downloading mirrors
Cannot update index (no local copy)
Downloading index
Waiting to acquire cache lock on /home/USER/.stack/pantry/hackage/hackage-security-lock
Acquired cache lock on /home/USER/.stack/pantry/hackage/hackage-security-lock
Released cache lock on /home/USER/.stack/pantry/hackage/hackage-security-lock
Updated package index downloaded
Calculating hashes to check for hackage-security rebases or filesystem changes
No old cache found, populating cache from scratch
Populating cache from file size 747617792, hash b0348cca768ad3ca54d4e72fa8279a0ac53bf42eee574d8f1ed253fccdf80b66
Populating package index cache ...
Processed 400 cabal files
[...]
Processed 151200 cabal files
Package index cache populated
[1 of 2] Compiling Main ( /home/USER/.stack/setup-exe-src/setup-mPHDZzAJ.hs, /home/USER/.stack/setup-exe-src/setup-mPHDZzAJ.o )
[2 of 2] Compiling StackSetupShim ( /home/USER/.stack/setup-exe-src/setup-shim-mPHDZzAJ.hs, /home/USER/.stack/setup-exe-src/setup-shim-mPHDZzAJ.o )
Linking /home/USER/.stack/setup-exe-cache/x86_64-linux-tinfo6/tmp-Cabal-simple_mPHDZzAJ_3.2.1.0_ghc-8.10.6 ...
kotlin-connectors-docu-generation-0.1.0.0: unregistering
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/basement-0.0.12/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/blaze-builder-0.4.2.1/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/base64-bytestring-1.1.0.0/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/base16-bytestring-1.0.1.0/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/base-orphans-0.8.4/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/base-compat-0.11.2/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/SHA-1.6.4.4/build
No directory could be located matching the supplied path: /tmp/stack-1ac9175b062b9926/HsYAML-0.2.1.0/build
I'm building a project inside a docker container, with this cmd:
stack shows this output before exit:
After reading
stack
code, I found aboutcheckOwnership
function, so I tried this:It seems to work ok (at least it started to compile dependencies).
Is this an expected behaviour?
I'm using this
stack
version inside haskell:7.10.3 docker image: