[BUG] failed to register layer: operation not permitted

dr460nf1r3 commented 9 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior

Newer versions of the container (>27.0.1.2) can't be pulled successfully anymore, failing with:

[root@docker:/var/garuda/docker-compose-runner/all-in-one]# docker compose up -d
[+] Running 9/10
 ⠧ nextcloud_app 9 layers [⣿⣿⣿⣿⣿⣿⣿⣿⣿]      0B/0B      Pulling                                                                5.7s 
   ✔ efb124c083f2 Download complete                                                                                          0.8s 
   ✔ b81e0d862760 Download complete                                                                                          0.4s 
   ✔ d0f59184f0b9 Download complete                                                                                          0.4s 
   ✔ af23b7546513 Download complete                                                                                          0.9s 
   ✔ c56105954327 Download complete                                                                                          0.8s 
   ✔ 029389a26518 Download complete                                                                                          1.3s 
   ✔ f7da7bb6051d Download complete                                                                                          1.2s 
   ✔ 34f31030b02d Download complete                                                                                          4.3s 
   ✔ 41970a3fd4e3 Download complete                                                                                          1.6s 
failed to register layer: operation not permitted

Since older versions (exemplary 27.0.1.2) are still getting pulled correctly and all other containers on this server also don't act up like that, I suspect something in the image is not working as expected.

I sadly also can't use the older working image due to a mismatch of versions:

Can't start Nextcloud because the version of the data (27.0.2.1) is higher than the docker image version (27.0.1.2) and downgrading is not supported. Are you sure you have pulled the newest image version?

The existing images and layers had already been cleared before creating this issue - one time via docker system prune -a and another time via completely wiping /var/lib/docker.

Following the output of docker info:

Client:               
 Version:    24.0.5                                 
 Context:    default
 Debug Mode: false   
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /nix/store/c9dxg39wxk8g77bzanmxxyag6gvml24r-docker-plugins/libexec/docker/cli-plugins/dock
er-buildx                                           
  compose: Docker Compose (Docker Inc.)
    Version:  2.21.0
    Path:     /nix/store/c9dxg39wxk8g77bzanmxxyag6gvml24r-docker-plugins/libexec/docker/cli-plugins/dock
er-compose   

Server:
 Containers: 20   
  Paused: 0
  Stopped: 0
 Images: 20
 Server Version: 24.0.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: journald
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: v1.7.5
 runc version: 
 init version: 
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.4
 Operating System: NixOS 23.11 (Tapir)
 OSType: linux
 Architecture: x86_64
 CPUs: 32
 Total Memory: 125.5GiB
 Name: docker
 ID: 6c36be33-6060-45ab-9be3-ebe8ae694799
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: true

WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

I'd appreciate help, even if it's just a hint on getting the older image to work with the newer data version. Thanks, also for providing and maintaining this image! 😊

Expected Behavior

The image can be pulled successfully.

Steps To Reproduce

I suppose you could try pulling the image with the config used:

  nextcloud_app:
    image: linuxserver/nextcloud:27.1.1
    container_name: nextcloud
    depends_on: [nextcloud_db, nextcloud_redis]
    environment:
      PUID: 1000
      PGID: 1000
      TZ: Europe/Berlin
    ports: [443:443]
    volumes: [./nextcloud/config:/config, ./nextcloud/data:/data]
    restart: always

Environment

- OS: NixOS-unstable
- How docker service was installed: via the recommended NixOS way `virtualisation.docker.enable = true;` ([link](https://search.nixos.org/options?channel=23.05&show=virtualisation.docker.enable&from=0&size=50&sort=relevance&type=packages&query=docker))

CPU architecture

x86-64

Docker creation

The config we were using before the breakage occurred can be found here.

`docker-compose up -d`

Container logs

No logs are available since the error occurs while pulling the image.

github-actions[bot] commented 9 months ago

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

nemchik commented 9 months ago

Can you docker pull linuxserver/nextcloud:27.1.1 and then rerun docker-compose up -d

dr460nf1r3 commented 9 months ago

[root@docker:/var/garuda/docker-compose-runner/all-in-one]# docker pull linuxserver/nextcloud:27.1.1
27.1.1: Pulling from linuxserver/nextcloud
253722c5a7d8: Extracting [==================================================>]   5.63MB/5.63MB
07a0e16f7be1: Download complete 
f93399c9350d: Download complete 
76fb3bb5b225: Download complete 
7bb40d29e696: Download complete 
bb4474cecf2a: Download complete 
2c651e8d9fb8: Download complete 
b9421bbf9836: Download complete 
d77cfece025b: Download complete 
failed to register layer: operation not permitted

Sadly also failing with the same error message.

nemchik commented 8 months ago

[root@docker:/var/garuda/docker-compose-runner/all-in-one]# docker pull linuxserver/nextcloud:27.1.1
27.1.1: Pulling from linuxserver/nextcloud
253722c5a7d8: Extracting [==================================================>]   5.63MB/5.63MB
07a0e16f7be1: Download complete 
f93399c9350d: Download complete 
76fb3bb5b225: Download complete 
7bb40d29e696: Download complete 
bb4474cecf2a: Download complete 
2c651e8d9fb8: Download complete 
b9421bbf9836: Download complete 
d77cfece025b: Download complete 
failed to register layer: operation not permitted

Sadly also failing with the same error message.

Sorry for the delayed response. This does not look like an issue with the image. It could possibly be a connectivity or DNS issue, or an issue with a specific registry. Here are some alternatives you can try:

docker pull lscr.io/linuxserver/nextcloud:27.1.1
docker pull ghcr.io/linuxserver/nextcloud:27.1.1
docker pull quay.io/linuxserver.io/nextcloud:27.1.1
docker pull registry.gitlab.com/linuxserver.io/docker-nextcloud/nextcloud:27.1.1

If any one of those work then you can use that in your compose. If none of them work then I would recommend trying to pull another image of ours to see if it's specific to nextcloud.

dr460nf1r3 commented 8 months ago

Sorry for the delayed response

Np, I'm guilty of it too!

If none of them work then I would recommend trying to pull another image of ours to see if it's specific to nextcloud.

Sadly all of them failed so I proceeded to try other ones:

[root@docker:~]# docker pull linuxserver/nginx
Using default tag: latest
latest: Pulling from linuxserver/nginx
a43e029ba662: Extracting [==================================================>]   5.63MB/5.63MB
07a0e16f7be1: Download complete
e3303cebae64: Download complete
35c9530ef606: Download complete
693b3eff61d8: Download complete
61906b476bf9: Download complete
505ce5c161b3: Download complete
86680f0a4aaf: Download complete
failed to register layer: operation not permitted

Same thing here:

[root@docker:~]# docker pull linuxserver/sqlitebrowser
Using default tag: latest
latest: Pulling from linuxserver/sqlitebrowser
a43e029ba662: Extracting [==================================================>]   5.63MB/5.63MB
07a0e16f7be1: Download complete
e3303cebae64: Download complete
35c9530ef606: Download complete
693b3eff61d8: Download complete
8f2dc51c48c4: Download complete
a88172b3d190: Download complete
8d1dc2303fd5: Download complete
90d408ea59dc: Download complete
dea2770dbc14: Download complete
ec46e0fd650f: Download complete
failed to register layer: operation not permitted

But I found one working among the ones I tried:

[root@docker:~]# docker pull linuxserver/sonarr:latest
latest: Pulling from linuxserver/sonarr
f83c38a97647: Pull complete
07a0e16f7be1: Pull complete
2f5368079c48: Pull complete
727f1a78792f: Pull complete
472bb230a37d: Pull complete
1141a882de05: Pull complete
c61c25f07bd5: Pull complete
bf12357cc297: Pull complete
Digest: sha256:296e553380b1b5d32be623ce2b97f67b69fc57449b9ef07195baa6a5df2b9bd6
Status: Downloaded newer image for linuxserver/sonarr:latest
docker.io/linuxserver/sonarr:latest

Hope this might give some ideas. As I said, every other image we run on the same server (lots) is working just fine.

j0nnymoe commented 8 months ago

Could be something related to nixOS, I'm not seeing reports for any other distro.

LinuxServer-CI commented 7 months ago

This issue has been automatically marked as stale because it has not had recent activity. This might be due to missing feedback from OP. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 4 months ago

This issue is locked due to inactivity

linuxserver / docker-nextcloud