checkpoint-restore / criu

Checkpoint/Restore tool
criu.org
Other
2.87k stars 582 forks source link

cannot restore vnc gui application launched by normal user. #2478

Open coldbloodx opened 6 days ago

coldbloodx commented 6 days ago

Description cannot restore vnc gui application launched by normal user.

  1. as normal user, e.g. leo, launch an gui application e.g. xclock via vncserver.sh(offered by criu)
  2. as root user, dump application started in step 1
  3. as root user, restore application with image dumpped in step 2 --> the application CANNOT be restored successfully.

Steps to reproduce the issue:

  1. create vncserver.sh like below:
    
    [leo@laworks 4cpu]$ cat vncserver.sh
    #!/bin/bash
    set -m
    Xvnc :25 -v -geometry 1440x900 -interface 0.0.0.0 -SecurityTypes none &
    pid=$!
    trap "kill $pid; wait" EXIT
    sleep 3
    DISPLAY=:25 $@
2. launch `xclock` with above script with a normal user, e.g. leo.

[leo@laworks 4cpu]$ unshare -r -i ./vncserver.sh xclock Xvnc TigerVNC 1.13.1 - built Apr 22 2024 00:00:00 Copyright (C) 1999-2022 TigerVNC Team and many others (see README.rst) See https://www.tigervnc.org for information on TigerVNC. Underlying X server release 12011000

Sat Sep 14 17:54:58 2024 vncext: VNC extension running! vncext: Listening for VNC connections on 0.0.0.0 interface(s), port 5925 vncext: created VNC server for screen 0 Warning: Missing charsets in String to FontSet conversion


3. dump above application from another terminal with root user.

[root@laworks criutool]# pgrep vncserver.sh 3768777 [root@laworks criutool]# criu dump -t pgrep vncserver.sh -D temp --shell-job Warn (compel/arch/x86/src/lib/infect.c:367): Will restore 3768781 with interrupted system call [root@laworks criutool]# echo $? 0 --> dumpped successfully. [root@laworks criutool]#

4. restore application from image dumpped from step 3, get `Operation not permitted error` like below.

[root@laworks criutool]# criu restore -D temp -v4 --tcp-established -d -j ... ... (00.003315) 3768777: cg: Move into 2 (00.003327) 3768777: cg: -> unified//user.slice/user-0.slice/session-5505.scope/cgroup.procs (00.003329) 3768777: uns: calling userns_move (-1, 0) (00.003366) uns: daemon calls 0x4410c0 (3768777, -1, 0) (00.011876) 3768777: Calling restore_sid() for init (00.011932) 3768777: Error (criu/util.c:1551): Unable to open the proc file system: Operation not permitted --> !!!here!!! (00.011990) uns: calling exit_usernsd (-1, 1) (00.012046) uns: daemon calls 0x4823d0 (3768795, -1, 1) (00.012061) uns:- daemon exits w/ 0 (00.012801) Error (criu/cr-restore.c:1517): 3768777 killed by signal 9: Killed (00.012814) uns: daemon stopped (00.012816) Error (criu/cr-restore.c:2557): Restoring FAILED. (00.013608) Error (criu/cgroup.c:1970): cg: cgroupd: recv req error: No such file or directory

here is full log:
[restore.log](https://github.com/user-attachments/files/17001838/restore.log)

**Describe the results you received:**
gui application started by normal user could not be restored

**Describe the results you expected:**
gui application started by normal user could be restored

**Additional information you deem important (e.g. issue happens only occasionally):**

**CRIU logs and information:**
here is full log:
[restore.log](https://github.com/user-attachments/files/17001838/restore.log)
<!--
You can either attach logs as files to the issue or put them under details
-->

<details><summary>CRIU full dump/restore logs:</summary>
<p>

[root@laworks criutool]# criu restore -D temp  -v4 --tcp-established -d -j
(00.000000) CRIU run id = 0xeffffffc003981db
(00.000030) Version: 3.19 (gitid 0)
(00.000035) Running on laworks Linux 5.14.0-427.28.1.el9_4.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Jul 31 15:28:35 UTC 2024 x86_64
(00.000062) Loaded kdat cache from /run/criu/criu.kdat
(00.000087) Hugetlb size 2 Mb is supported but cannot get dev's number
(00.000107) Hugetlb size 1024 Mb is supported but cannot get dev's number
(00.000769) Will dump/restore TCP connections
(00.000782) mnt-v2: Mounts-v2 requires MOVE_MOUNT_SET_GROUP support
(00.000790) Mount engine fallback to --mntns-compat-mode mode
(00.000803) rlimit: RLIMIT_NOFILE unlimited for self
(00.000932) cpu: x86_family 6 x86_vendor_id GenuineIntel x86_model_id Intel(R) Xeon(R) Platinum 8255C CPU @ 2.50GHz
(00.000944) cpu: fpu: xfeatures_mask 0x2e5 xsave_size 2696 xsave_size_max 2696 xsaves_size 2440
(00.000959) cpu: fpu: x87 floating point registers     xstate_offsets      0 / 0      xstate_sizes    160 / 160
(00.000962) cpu: fpu: AVX registers                    xstate_offsets    576 / 576    xstate_sizes    256 / 256
(00.000969) cpu: fpu: AVX-512 opmask                   xstate_offsets   1088 / 832    xstate_sizes     64 / 64
(00.000971) cpu: fpu: AVX-512 Hi256                    xstate_offsets   1152 / 896    xstate_sizes    512 / 512
(00.000973) cpu: fpu: AVX-512 ZMM_Hi256                xstate_offsets   1664 / 1408   xstate_sizes   1024 / 1024
(00.000974) cpu: fpu: Protection Keys User registers   xstate_offsets   2688 / 2432   xstate_sizes      8 / 8
(00.000977) cpu: fpu:1 fxsr:1 xsave:1 xsaveopt:1 xsavec:1 xgetbv1:1 xsaves:1
(00.001006) kernel pid_max=4194304
(00.001012) Reading image tree
(00.001037) Add mnt ns 6 pid 3768777
(00.001044) Add net ns 2 pid 3768777
(00.001045) Add pid ns 1 pid 3768777
(00.001063) pstree pid_max=3768781
(00.001073) Migrating process tree (SID 3750821->3674489)
(00.001076) Will restore in 18000000 namespaces
(00.001077) NS mask to use 18000000
(00.001104) Collecting 51/56 (flags 3)
(00.001113) No memfd.img image
(00.001115)  `- ... done
(00.001118) Collecting 40/54 (flags 2)
(00.001131) Collected [usr/bin/bash] ID 0x1
(00.001134) Collected [usr/lib/locale/locale-archive] ID 0x2
(00.001136) Collected [usr/lib64/libc.so.6] ID 0x3
(00.001140) Collected [usr/lib64/libtinfo.so.6.2] ID 0x4
(00.001142) Collected [usr/lib64/gconv/gconv-modules.cache] ID 0x5
(00.001144) Collected [usr/lib64/ld-linux-x86-64.so.2] ID 0x6
(00.001145) Collected [dev/pts/0] ID 0x8
(00.001151) Collected [home/leo/criutool/4cpu/vncserver.sh] ID 0x9
(00.001157) Collected [home/leo/criutool/4cpu] ID 0xa
(00.001159) Collected [.] ID 0xb
(00.001163) Collected [usr/bin/Xvnc] ID 0xc
(00.001165) Collected [usr/share/fonts/liberation-mono/LiberationMono-Bold.ttf] ID 0xd
(00.001167) Collected [usr/lib64/libedit.so.0.0.64] ID 0xe
(00.001169) Collected [usr/lib64/libzstd.so.1.5.1] ID 0xf
(00.001171) Collected [usr/lib64/libLLVM-17.so] ID 0x10
(00.001173) Collected [usr/lib64/libdrm_nouveau.so.2.0.0] ID 0x11
(00.001175) Collected [usr/lib64/libelf-0.190.so] ID 0x12
(00.001176) Collected [usr/lib64/libdrm_amdgpu.so.1.0.0] ID 0x13
(00.001182) Collected [usr/lib64/libdrm_radeon.so.1.0.1] ID 0x14
(00.001193) Collected [usr/lib64/libexpat.so.1.8.10] ID 0x15
(00.001196) Collected [usr/lib64/libdrm.so.2.4.0] ID 0x16
(00.001199) Collected [usr/lib64/dri/swrast_dri.so] ID 0x17
(00.001201) Collected [usr/lib64/libxcb-dri3.so.0.0.0] ID 0x18
(00.001203) Collected [usr/lib64/libglapi.so.0.0.0] ID 0x19
(00.001206) Collected [usr/lib64/libnss_sss.so.2] ID 0x1a
(00.001207) Collected [usr/lib64/libpcre.so.1.2.12] ID 0x1b
(00.001210) Collected [usr/lib64/libbrotlicommon.so.1.0.9] ID 0x1c
(00.001211) Collected [usr/lib64/libgraphite2.so.3.2.1] ID 0x1d
(00.001219) Collected [usr/lib64/libglib-2.0.so.0.6800.4] ID 0x1e
(00.001221) Collected [usr/lib64/libxcb.so.1.1.0] ID 0x1f
(00.001225) Collected [usr/lib64/libbrotlidec.so.1.0.9] ID 0x20
(00.001227) Collected [usr/lib64/libharfbuzz.so.0.20704.0] ID 0x21
(00.001233) Collected [usr/lib64/libGLdispatch.so.0.0.0] ID 0x22
(00.001236) Collected [usr/lib64/libX11.so.6.4.0] ID 0x23
(00.001238) Collected [usr/lib64/libfreetype.so.6.17.4] ID 0x24
(00.001240) Collected [usr/lib64/libunistring.so.2.1.0] ID 0x25
(00.001243) Collected [usr/lib64/libpng16.so.16.37.0] ID 0x26
(00.001253) Collected [usr/lib64/libbz2.so.1.0.8] ID 0x27
(00.001258) Collected [usr/lib64/libffi.so.8.1.0] ID 0x28
(00.001260) Collected [usr/lib64/libcap-ng.so.0.0.0] ID 0x29
(00.001262) Collected [usr/lib64/libp11-kit.so.0.3.1] ID 0x2a
(00.001263) Collected [usr/lib64/libstdc++.so.6.0.29] ID 0x2b
(00.001265) Collected [usr/lib64/libXext.so.6.4.0] ID 0x2c
(00.001267) Collected [usr/lib64/libGLX.so.0.0.0] ID 0x2d
(00.001271) Collected [usr/lib64/libfontenc.so.1.0.0] ID 0x2e
(00.001274) Collected [usr/lib64/libidn2.so.0.3.7] ID 0x2f
(00.001277) Collected [usr/lib64/libm.so.6] ID 0x30
(00.001279) Collected [usr/lib64/libGL.so.1.7.0] ID 0x31
(00.001281) Collected [usr/lib64/libgnutls.so.30.37.1] ID 0x32
(00.001283) Collected [usr/lib64/libtasn1.so.6.6.0] ID 0x33
(00.001284) Collected [usr/lib64/libeconf.so.0.4.1] ID 0x34
(00.001286) Collected [usr/lib64/libaudit.so.1.0.0] ID 0x35
(00.001289) Collected [usr/lib64/libgcc_s-11-20231218.so.1] ID 0x36
(00.001291) Collected [usr/lib64/libXdmcp.so.6.0.0] ID 0x37
(00.001293) Collected [usr/lib64/libXau.so.6.0.0] ID 0x38
(00.001296) Collected [usr/lib64/libXfont2.so.2.0.0] ID 0x39
(00.001298) Collected [usr/lib64/libpixman-1.so.0.40.0] ID 0x3a
(00.001300) Collected [usr/lib64/libnettle.so.8.8] ID 0x3b
(00.001302) Collected [usr/lib64/libz.so.1.2.11] ID 0x3c
(00.001304) Collected [usr/lib64/libgmp.so.10.4.0] ID 0x3d
(00.001306) Collected [usr/lib64/libhogweed.so.6.8] ID 0x3e
(00.001309) Collected [usr/lib64/libpam.so.0.85.1] ID 0x3f
(00.001332) Collected [usr/lib64/libjpeg.so.62.3.0] ID 0x40
(00.001341) epoll: Collected eventpoll: id 0x000041 flags 0x02
(00.001353) unix:  `- Got id 0x42 ino 45542911 type SOCK_STREAM state TCP_LISTEN peer 0 (name @/tmp/.X11-unix/X25 dir -)
(00.001366) unix:  `- Got id 0x43 ino 45542912 type SOCK_STREAM state TCP_LISTEN peer 0 (name /tmp/.X11-unix/X25 dir -)
(00.001373) Collected [home/leo/criutool/4cpu] ID 0x46
(00.001377) Collected [.] ID 0x47
(00.001378) Collected [usr/bin/xclock] ID 0x48
(00.001380) Collected [usr/share/fonts/dejavu-sans-fonts/DejaVuSans.ttf] ID 0x49
(00.001382) Collected [usr/lib/fontconfig/cache/123d59b33ddb0e7c76bb24004bd5cfac-le64.cache-8] ID 0x4a
(00.001386) Collected [usr/lib/fontconfig/cache/3f821257dd33660ba7bbb45c32deb84c-le64.cache-8] ID 0x4b
(00.001390) Collected [usr/lib/fontconfig/cache/131ab5cc1583381c4f7ce0194912c56d-le64.cache-8] ID 0x4c
(00.001392) Collected [usr/lib/fontconfig/cache/26078b1cf62d7535e9fc9c56a8803883-le64.cache-8] ID 0x4d
(00.001395) Collected [usr/lib/fontconfig/cache/ac68f755438cc3dc5a526084839fc7ca-le64.cache-8] ID 0x4e
(00.001397) Collected [usr/lib/fontconfig/cache/f951a6bc01c50d58ac4af16a0108457e-le64.cache-8] ID 0x4f
(00.001400) Collected [usr/lib/fontconfig/cache/6b4d77390f008fe4d7fb61c915674aee-le64.cache-8] ID 0x50
(00.001403) Collected [usr/lib/fontconfig/cache/bf4088b6c6290c8d6936483b844e6a40-le64.cache-8] ID 0x51
(00.001405) Collected [usr/lib/fontconfig/cache/f132fa2327207a6ac3298c0518879731-le64.cache-8] ID 0x52
(00.001410) Collected [usr/lib/fontconfig/cache/b887eea8f1b96e1d899b44ed6681fc27-le64.cache-8] ID 0x53
(00.001414) Collected [usr/lib/fontconfig/cache/860639f272b8b4b3094f9e399e41bccd-le64.cache-8] ID 0x54
(00.001416) Collected [usr/lib/fontconfig/cache/5d33f04e74a97395cf88bbd83847f1f1-le64.cache-8] ID 0x55
(00.001425) Collected [usr/lib/fontconfig/cache/df893b4576ad6107f9397134092c4059-le64.cache-8] ID 0x56
(00.001427) Collected [usr/lib/fontconfig/cache/900402270e15d763a6e008bb2d4c7686-le64.cache-8] ID 0x57
(00.001431) Collected [usr/lib/fontconfig/cache/47f48679023f44a4d1e44699a69464f6-le64.cache-8] ID 0x58
(00.001433) Collected [usr/lib/fontconfig/cache/2881ed3fd21ca306ddad6f9b0dd3189f-le64.cache-8] ID 0x59
(00.001435) Collected [usr/lib/fontconfig/cache/3c3fb04d32a5211b073874b125d29701-le64.cache-8] ID 0x5a
(00.001436) Collected [usr/lib/fontconfig/cache/3e9ca894d7ccd8b9fedb236c4f3f7c4e-le64.cache-8] ID 0x5b
(00.001440) Collected [usr/lib/fontconfig/cache/5535e07303e0edee0923e77e4e59b69c-le64.cache-8] ID 0x5c
(00.001443) Collected [usr/lib/fontconfig/cache/cf759820c416606818fc74e5e9991313-le64.cache-8] ID 0x5d
(00.001446) Collected [usr/lib/fontconfig/cache/e34b99a1e22e6f7451938fb9934274e6-le64.cache-8] ID 0x5e
(00.001448) Collected [usr/lib/fontconfig/cache/d63f98f14a274bd69a5425fc33aaac6b-le64.cache-8] ID 0x5f
(00.001452) Collected [usr/lib/fontconfig/cache/8810ee51c158c7bfaed726592ffe4eb9-le64.cache-8] ID 0x60
(00.001454) Collected [usr/lib/fontconfig/cache/7ee6df7a8311986241317a58487e0145-le64.cache-8] ID 0x61
(00.001455) Collected [usr/lib/fontconfig/cache/6ee3103884cce7b2fe6f32eba9089175-le64.cache-8] ID 0x62
(00.001457) Collected [usr/lib/fontconfig/cache/7bbebb41f246c24642924bd8585d5345-le64.cache-8] ID 0x63
(00.001462) Collected [usr/lib/fontconfig/cache/221930ae9526a9cb8049af2916f03412-le64.cache-8] ID 0x64
(00.001466) Collected [usr/lib/fontconfig/cache/6ba42ae0000f58711b5caaf10d690066-le64.cache-8] ID 0x65
(00.001468) Collected [usr/lib64/libXfixes.so.3.1.0] ID 0x66
(00.001474) Collected [usr/lib64/libXcursor.so.1.0.2] ID 0x67
(00.001476) Collected [usr/lib64/liblzma.so.5.2.5] ID 0x68
(00.001477) Collected [usr/lib64/libxml2.so.2.9.13] ID 0x69
(00.001482) Collected [usr/lib/fontconfig/cache/863140a4aaae38446c3fb212df9b1ab5-le64.cache-8] ID 0x6a
(00.001488) Collected [usr/lib64/libuuid.so.1.3.0] ID 0x6b
(00.001490) Collected [usr/lib64/libfontconfig.so.1.12.0] ID 0x6c
(00.001494) Collected [usr/lib64/libICE.so.6.3.0] ID 0x6d
(00.001496) Collected [usr/lib64/libSM.so.6.0.1] ID 0x6e
(00.001498) Collected [usr/lib64/libXpm.so.4.11.0] ID 0x6f
(00.001500) Collected [usr/lib64/libxkbfile.so.1.0.2] ID 0x70
(00.001501) Collected [usr/lib64/libXft.so.2.3.3] ID 0x71
(00.001503) Collected [usr/lib64/libXrender.so.1.3.0] ID 0x72
(00.001505) Collected [usr/lib64/libXt.so.6.0.0] ID 0x73
(00.001508) Collected [usr/lib64/libXmu.so.6.2.0] ID 0x74
(00.001510) Collected [usr/lib64/libXaw7.so.7.0.0] ID 0x75
(00.001513) Collected [usr/lib/fontconfig/cache/22f06f3be2d16d058da85b73ae1dc5b1-le64.cache-8] ID 0x76
(00.001515) Collected [usr/lib/fontconfig/cache/427eb62078a821f08aa6ed364f2467bf-le64.cache-8] ID 0x77
(00.001517) Collected [usr/lib/fontconfig/cache/210c0516121708a580e22e6b1f9a103a-le64.cache-8] ID 0x78
(00.001519) Collected [usr/lib/fontconfig/cache/b14e78aa9400ae7a28193faee1d62280-le64.cache-8] ID 0x79
(00.001521) unix:  `- Got id 0x7a ino 45543803 type SOCK_STREAM state TCP_ESTABLISHED peer 45542916 (name - dir -)
(00.001525) unix:  `- Got id 0x45 ino 45542916 type SOCK_STREAM state TCP_ESTABLISHED peer 45543803 (name @/tmp/.X11-unix/X25 dir -)
(00.001527) Collected [home/leo/criutool/4cpu] ID 0x7b
(00.001529) Collected [.] ID 0x7c
(00.001532)  `- ... done
(00.001533) Collecting 46/68 (flags 0)
(00.001536) No remap-fpath.img image
(00.001538)  `- ... done
(00.001565) No apparmor.img image
(00.001591) cg: Preparing cgroups yard (cgroups restore mode 0x4)
(00.001870) cg: Opening .criu.cgyard.PSGh8p as cg yard
(00.001888) cg:         Making controller dir .criu.cgyard.PSGh8p/unified ()
(00.001920) cg: Determined cgroup dir unified/user.slice/user-0.slice/session-5505.scope already exist
(00.001926) cg: Skip restoring properties on cgroup dir unified/user.slice/user-0.slice/session-5505.scope
(00.002191) Running pre-restore scripts
(00.002307) cg: cgroud: Daemon started
(00.002467) No pidns-1.img image
(00.002535) uns: Daemon started
(00.002580) Forking task with 3768777 pid (flags 0x18000000)
(00.002583) Creating process using clone3()
(00.002795) PID: real 3768777 virt 3768777
(00.002948) Wait until namespaces are created
(00.003169) 3768777: timens: monotonic -107 944345790
(00.003189) 3768777: timens: boottime -107 944325663
(00.003245) Running setup-namespaces scripts
(00.003315) 3768777: cg: Move into 2
(00.003327) 3768777: cg:   `-> unified//user.slice/user-0.slice/session-5505.scope/cgroup.procs
(00.003329) 3768777: uns: calling userns_move (-1, 0)
(00.003366) uns: daemon calls 0x4410c0 (3768777, -1, 0)
(00.011876) 3768777: Calling restore_sid() for init
(00.011932) 3768777: Error (criu/util.c:1551): Unable to open the proc file system: Operation not permitted
(00.011990) uns: calling exit_usernsd (-1, 1)
(00.012046) uns: daemon calls 0x4823d0 (3768795, -1, 1)
(00.012061) uns: `- daemon exits w/ 0
(00.012801) Error (criu/cr-restore.c:1517): 3768777 killed by signal 9: Killed
(00.012814) uns: daemon stopped
(00.012816) Error (criu/cr-restore.c:2557): Restoring FAILED.
(00.013608) Error (criu/cgroup.c:1970): cg: cgroupd: recv req error: No such file or directory

</p>
</details>

<details><summary>Output of `criu --version`:</summary>
<p>

[root@laworks criutool]# criu --version Version: 3.19


</p>
</details>

<details><summary>Output of `criu check --all`:</summary>
<p>

[root@laworks criutool]# criu check --all Warn (criu/cr-check.c:1346): Nftables based locking requires libnftables and set concatenations support Looks good but some kernel features are missing which, depending on your process tree, may cause dump or restore failure.



</p>
</details>

**Additional environment details:**
adrianreber commented 6 days ago

Just curious why you are using a user namespace (unshare -r implies --user)? You do not explicitly mention the usage of a user namespace in your description. Have you tried it without a user namespace?

coldbloodx commented 2 days ago

If I did not create a user ns, unshare will report operation not permitted like below

[leo@laworks 4cpu]$ unshare -i ./vncserver.sh xclock
unshare: unshare failed: Operation not permitted  
coldbloodx commented 2 days ago

just have another try with newns provided by this link: https://criu.org/VNC it cannot create ipc namespace either with normal user.

[leo@laworks 4cpu]$ ./newns ./vncserver.sh xclock
clone() failed: Operation not permitted
[leo@laworks 4cpu]$ ll
total 1968
-rwxr-xr-x 1 leo leo     491 Sep 14 14:42 4cpu.sh
-rw-r--r-- 1 leo leo      58 Sep 14 14:42 clean.sh
-rw-r--r-- 1 leo leo 1970243 Sep 14 14:42 fluent-test.cas
-rw-r--r-- 1 leo leo      32 Sep 14 14:42 hostfile.4cpu
-rw-r--r-- 1 leo leo     158 Sep 14 14:42 journal
-rwxr-xr-x 1 leo leo   18040 Sep 19 10:02 newns
-rwxr-xr-x 1 leo leo     149 Sep 14 14:42 vncserver.sh
[leo@laworks 4cpu]$

any ideas? how could I work around this with a normal user? @adrianreber