search

ibm-messaging / mq-container

Container images for IBM® MQ
Apache License 2.0
254 stars 189 forks source link

Bind-mount problem when building on Windows 10 #269

Open pk1972 opened 5 years ago

pk1972 commented 5 years ago

As a beginner in Docker i fear i'm missing some vital points here. Followed instructions in 'Building.md' for making a dev-server. Got the infamous 'gzip:stdin:not in gzip format ' error as well. Found the post describing chmod detail, added world permissions to archive but still getting this: '+ false

gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error is not recoverable: exiting now The command '/bin/sh -c chmod u+x /usr/local/bin/install-mq.sh && sleep 1 && install-mq.sh && rm -rf /var/mqm && /opt/mqm/bin/crtmqdir -f -s' returned a non-zero code: 2 2019/01/30 07:25:03 copy from pipe to stdout failed: write /dev/stdout: The specified network name is no longer available. build-server build docker build --build-arg BASE_IMAGE=mq-sdk:9.1.1.0-x86_64-ubuntu-16.04 -t mq-golang-sdk:9.1.1.0-x86_64-ubuntu-16.04 -f incubating/mq-golang-sdk/Dockerfile . Sending build context to Docker daemon 10.9MB Step 1/7 : ARG BASE_IMAGE=mq-sdk:9.1.1.0-x86_64-ubuntu-16.04 Step 2/7 : FROM $BASE_IMAGE pull access denied for mq-sdk, repository does not exist or may require 'docker login' Makefile-UBUNTU:267: recipe for target 'build-golang-sdk-ex' failed make[1]: [build-golang-sdk-ex] Error 1 make[1]: Leaving directory '/root/mq-container' Makefile:73: recipe for target 'build-devserver-ubuntu' failed make: [build-devserver-ubuntu] Error 2'

Propably missing something obvious, but i'm blind to what obviously.

arthurbarr commented 5 years ago

I wonder if the tar.gz hasn't downloaded properly. My copy of mqadv_dev911_ubuntu_x86-64.tar.gz is 839,931,811 bytes. Could you perhaps check your copy, and if it isn't that size, just delete it, and the build will download it again.

If that's not it, it could be related to something on your machine, so some details of how you're running Docker would be useful.

pk1972 commented 5 years ago

Tar.gz downloaded ok, checked that . Running Docker on WSL. My Docker info: :~/mq-container# docker info Containers: 85 Running: 30 Paused: 0 Stopped: 55 Images: 44 Server Version: 18.09.1 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: 9754871865f7fe2f4e74d43e2fc7ccd237edcbce runc version: 96ec2177ae841256168fcf76954f7177af9446eb init version: fec3683 Security Options: seccomp Profile: default Kernel Version: 4.9.125-linuxkit Operating System: Docker Desktop OSType: linux Architecture: x86_64 CPUs: 2 Total Memory: 1.934GiB Name: docker-desktop ID: LY26:FDZA:CL56:ONZH:NX7G:B7KE:WCRR:G53L:IS3D:PGEK:LPSS:KHCD Docker Root Dir: /var/lib/docker Debug Mode (client): false Debug Mode (server): true File Descriptors: 121 Goroutines: 124 System Time: 2019-01-30T13:29:50.7090629Z EventsListeners: 1 Username: pk1972 Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Product License: Community Engine

pk1972 commented 5 years ago

Tried setting $MQ_ARCHIVE_DEV to the downloaded mqadv_dev911_ubuntux86-64.tar.gz also but same error happens. "_+ false

gzip: stdin: not in gzip format tar: Child returned status 1 tar: Error is not recoverable: exiting now"__

arthurbarr commented 5 years ago

If you're using WSL, you need to look at this guide by Microsoft to get bind-mounts working. There's also a link to this in our doc.

pk1972 commented 5 years ago

@arthurbarr I tried that way first. But since that produced the same result i went for the method posted in here:https://nickjanetakis.com/blog/setting-up-docker-for-windows-and-wsl-to-work-flawlessly Which is what i run now. So for me it fails on both methods.

pk1972 commented 5 years ago

Wiped my WSL installation, built it from scratch according to instructions provided in the docs Running Docker works fine, but to build a queuemanager with 9.1.1 still fails with the same error. I must be missing something fundamental here.

arthurbarr commented 5 years ago

Can you use a bind-mounted volume with Docker? For example:

echo "Hello world" > /tmp/host.file
docker run --rm -it --volume /tmp/host.file:/var/container.file ubuntu:16.04 cat /var/container.file

That should print "Hello world" if bind-mounting volumes is working OK.

The build bind-mounts the tar file into a Docker container, then hosts that tar.gz on an HTTP server, on a local network. The main build container runs (as part of docker build), which uses curl to download the tar file. It does this to ensure the tar file doesn't get committed into a Docker image layer. The trouble is that the curl command appears to only be downloading 153 bytes, instead of the expected 801 MB. So I suspect that the tar file isn't making it into the HTTP server container.

pk1972 commented 5 years ago

I think i have an issue with the bind-mounting. Ran your sample and the result was: cat: /var/container.file: Is a directory

arthurbarr commented 5 years ago

I think that might happen if Docker can't find the file on the host. The volume in the bind-mount needs to be a fully qualified path.

pk1972 commented 5 years ago

@arthurbarr Can you expand on that, "The volume in the bind-mount needs to be fully qualified path" ? I tried setting volume to a folder on mnt/c (which is accessible from docker, and writable) but i still get the error 'cat: /mnt/c/test/container.file: Is a directory'

arthurbarr commented 5 years ago

I mean that the host side of the bind mount directory (before the ":") needs to be a fully qualified path (not a relative path). It would help if you could supply the full command you are using. Based on your example file name there, you might try running this:

echo "Hello world" > /mnt/c/test/host.file
docker run --rm -it --volume /mnt/c/test/host.file:/var/container.file ubuntu:16.04 cat /var/container.file
pk1972 commented 5 years ago

Well, got an update for my Windows 10 (updated to 1809 build) and had to reinstall WSL completely. Did the WSL setup linked in Build Docs and still fails with the TAR archive. Ran your sample and here is the result: pk1972@SEPC4234:~/mq-container/downloads$ echo "Hello world" > /mnt/c/test/host.file

pk1972@SEPC4234:~/mq-container/downloads$ docker run --rm -it --volume /mnt/c/test/host.file:/var/container.file ubuntu:16.04 cat /var/container.file cat: /var/container.file: Is a directory

Maybe i should just do VMWare Linux and use docker from there instead ?

arthurbarr commented 5 years ago

I'm afraid that I don't know enough about WSL to be able to help you get that going. However, for all the apparent complexity of the mq-container build, it all boils down to a couple of regular docker build commands. We have extra wrapper code to set up an HTTP server to host the MQ tar.gz file, but if you're happy doing that yourself, you could give this a go.

Firstly, start an HTTP server to host the tar.gz file. Here is a way of doing this with Python, which serves up the contents of the current directory on port 8000:

python -m SimpleHTTPServer 8000 .

Next, build the MQ Go SDK image, which is used to build the Go programs with the MQ client:

docker build -t mq-golang-sdk -f incubating/mq-golang-sdk/Dockerfile .

Next, get your TCP/IP address for the host you're running that HTTP server on (i.e. your Windows machine), then plug it into this command (or just set MY_IP_ADDRESS):

docker build \
  --tag mq \
  --file Dockerfile-server \
  --build-arg MQ_URL=http://${MY_IP_ADDRESS}:8000/downloads/mqadv_dev911_ubuntu_x86-64.tar.gz \
  --build-arg BASE_IMAGE=ubuntu:16.04 \
  --build-arg BUILDER_IMAGE=mq-golang-sdk \