search

hepsw / docks

a set of Dockerfiles defining docker containers for HEP software and appliances.
BSD 3-Clause "New" or "Revised" License
17 stars 13 forks source link

docker pull cvmfs-atlas hangs on layer d696c0317884 #12

Closed lukasheinrich closed 8 years ago

lukasheinrich commented 8 years ago

is this an issue with the layer? I've seen this form a number of different dockerhosts

Using default tag: latest
latest: Pulling from hepsw/cvmfs-atlas
1b94336bbed4: Download complete 
ea9cd6ca7a80: Download complete 
e011fbdbe0c7: Download complete 
e9d62d68ffe6: Download complete 
4dd3cf3b59da: Download complete 
143f9ff3ef0e: Download complete 
b7ce03b9dd6b: Download complete 
d696c0317884: Downloading [======================>                            ] 81.63 MB/178.4 MB
a50e325115c3: Download complete 
baae8e3bd35c: Download complete 
ec3cf0ae4771: Download complete 
4c3ae5af6643: Download complete 
f9a4d8201952: Download complete 
9aff04fd6f24: Download complete 
ab8193113763: Download complete 
bb2a60b9f538: Download complete 
d295906fd39d: Download complete 
fb683116aa5a: Download complete 
0b094b06f8e3: Download complete 
8d5302a96d3f: Download complete 
2380aad76d22: Download complete 
6b76f7a355de: Download complete 
e6b39308cf7f: Download complete 
2b3290a77736: Download complete 
3bb10d9ba474: Download complete 
e392e3380945: Download complete 
7371dd17d1c3: Download complete 
e4c881848adf: Download complete 
f24a512654a9: Download complete 
36b8ccfed8a9: Download complete 
a0bbc80e08be: Download complete 
b3dbc1a4e8cc: Download complete 
4bbfc0b24ffe: Download complete 
4bbfc0b24ffe: Layer already being pulled by another client. Waiting.
sbinet commented 8 years ago

hum... which cvmfs-atlas version are you trying to fetch?

I just zapped away all of my cvmfs- images and fetched cvmfs-atlas anew:

$> docker images | grep cvmfs-
$> docker pull hepsw/cvmfs-atlas
Using default tag: latest
latest: Pulling from hepsw/cvmfs-atlas
5be847e36d30: Pull complete 
64f72a6c75de: Pull complete 
f450f56ea6d1: Pull complete 
f83e45757e65: Pull complete 
8d27ab9d270e: Pull complete 
6d06dcc8dada: Pull complete 
4620cdb0e98a: Pull complete 
8418873fea09: Pull complete 
19364785acc8: Pull complete 
007cc509948f: Pull complete 
da5daa1ccd4a: Pull complete 
ae35aba6c7bd: Pull complete 
aab189dfecf4: Pull complete 
749844409f6e: Pull complete 
50c87541e48a: Pull complete 
b51098b8947e: Pull complete 
7bb5cdcd007a: Pull complete 
6690512ba49c: Pull complete 
a2606166caae: Pull complete 
a30e8f9c8ce2: Pull complete 
607dabedd3de: Pull complete 
abbb7a7c8ec5: Pull complete 
2fd50c3afac0: Pull complete 
921a8e9d1a3b: Pull complete 
7eb47986838c: Pull complete 
fda9a5e8a2be: Pull complete 
cd0e76652a2f: Pull complete 
202de6f2e02e: Pull complete 
316e95e3300f: Pull complete 
413ceb6cc10c: Pull complete 
6e18121d4ef8: Pull complete 
Digest: sha256:92c0f3ec563ecb2a269207b633bca181757cf57888fd25d24e77641173a04656
Status: Downloaded newer image for hepsw/cvmfs-atlas:latest

$>  docker images | grep cvmfs-atlas
hepsw/cvmfs-atlas                     latest              6e18121d4ef8        3 months ago        499.8 MB

here is my docker infos, BTW:

$>  docker info
Containers: 0
Images: 251
Server Version: 1.9.1
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.6-1-ARCH
Operating System: Arch Linux (containerized)
CPUs: 4
Total Memory: 7.737 GiB
Name: clrinfopc07
ID: O4WK:GKZ2:IO4W:B4HJ:CNBH:DEPT:FVFF:CEL7:R5Z2:4AAP:5FXN:ALIP
Username: binet
Registry: https://index.docker.io/v1/

$> docker version
Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.5.3
 Git commit:   a34a1d5-dirty
 Built:        Sat Jan 16 15:43:56 UTC 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.5.1
 Git commit:   a34a1d5-dirty
 Built:        Sun Nov 22 00:15:15 UTC 2015
 OS/Arch:      linux/amd64
lukasheinrich commented 8 years ago

weird. so my docker installation on an CERN OpenStack machine (CC7) gives some more info. seems like it can't handle the tarbar: archive/tar: invalid tar header

Using default tag: latest
latest: Pulling from hepsw/cvmfs-atlas
27a7a1b1ef22: Download complete 
c45527e72f71: Download complete 
5be847e36d30: Download complete 
64f72a6c75de: Download complete 
f450f56ea6d1: Download complete 
f83e45757e65: Download complete 
8d27ab9d270e: Download complete 
6d06dcc8dada: Downloading [====================================>              ] 131.6 MB/178.4 MB
4620cdb0e98a: Download complete 
8418873fea09: Download complete 
19364785acc8: Download complete 
007cc509948f: Download complete 
da5daa1ccd4a: Download complete 
ae35aba6c7bd: Download complete 
aab189dfecf4: Download complete 
749844409f6e: Download complete 
50c87541e48a: Download complete 
b51098b8947e: Download complete 
7bb5cdcd007a: Download complete 
6690512ba49c: Download complete 
a2606166caae: Download complete 
a30e8f9c8ce2: Download complete 
607dabedd3de: Download complete 
abbb7a7c8ec5: Download complete 
2fd50c3afac0: Download complete 
921a8e9d1a3b: Download complete 
7eb47986838c: Download complete 
fda9a5e8a2be: Download complete 
cd0e76652a2f: Download complete 
202de6f2e02e: Download complete 
316e95e3300f: Download complete 
413ceb6cc10c: Download complete 
6e18121d4ef8: Download complete 
Pulling repository docker.io/hepsw/cvmfs-atlas
4bbfc0b24ffe: Error pulling image (latest) from docker.io/hepsw/cvmfs-atlas, ApplyLayer exit status 1 stdout:  stderr: archive/tar: invalid tar header 
1b94336bbed4: Error downloading dependent layers 
Error pulling image (latest) from docker.io/hepsw/cvmfs-atlas, ApplyLayer exit status 1 stdout:  stderr: archive/tar: invalid tar header

docker version

Client:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5
 Built:        Fri Nov 20 13:25:01 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.9.1
 API version:  1.21
 Go version:   go1.4.2
 Git commit:   a34a1d5
 Built:        Fri Nov 20 13:25:01 UTC 2015
 OS/Arch:      linux/amd64

here's my docker info

Containers: 43
Images: 30
Server Version: 1.9.1
Storage Driver: devicemapper
 Pool Name: docker-253:1-67137935-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 107.4 GB
 Backing Filesystem: 
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 6.681 GB
 Data Space Total: 107.4 GB
 Data Space Available: 31.88 GB
 Metadata Space Used: 14.51 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.133 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.93-RHEL7 (2015-01-28)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.10.0-229.4.2.el7.x86_64
Operating System: CentOS Linux 7 (Core)
CPUs: 2
Total Memory: 3.703 GiB
Name: lheinric-recast-dockerinteractive.cern.ch
ID: 6K2N:SRTH:TWQY:LZI6:IHVY:DQRX:CHIF:QNJS:24LO:LNKT:SSVK:BR3J
sbinet commented 8 years ago

my reading of this and looking around (such as here: https://github.com/docker/docker/issues/15561 which hints at this) would make me think you would need to remove and reimport the slc-base and cvmfs-xyz containers

lukasheinrich commented 8 years ago

removing all containers (or for that matter creating a new docker-machine alltogether) doesn't seem to help. I'll try rebuilding from the Dockerfile

lukasheinrich commented 8 years ago

rebuilding worked, maybe the images could be re-pushed to Docker Hub?

sbinet commented 8 years ago

I'll rebuild and re-push (but I probably need to do that not from my archlinux box, but from a centos7 one)

lukasheinrich commented 8 years ago

unfortunately even with a working cvmfs-atlas image, I can't cross-bind the volumes. While adding the corresponding VOLUME /cvmfs line to the docker file exposes it, somehow the FUSE mount does not work when accessing it from a different container. Binding a cvmfs mount from the host, though, does. Do you know if this is impossible by design?

ponyisi commented 8 years ago

This issue seems to somehow be preventing automatic build of a derived docker image on the Docker Hub (!). Will bother relevant people later...

sbinet commented 8 years ago

hi,

I believe this is fixed (I finally carved up some time to create a cozy little centos-7 VM+docker)

please re-open if you still have issues.