ruyisdk / ruyi

RuyiSDK Package Manager
Apache License 2.0
12 stars 5 forks source link

device provision fails to unpack image for Meles & LPi4A #128

Closed KevinMX closed 3 months ago

KevinMX commented 3 months ago
mx @ Phony in ~ |15:37:30  
$ RUYI_DEBUG=X ruyi device provision
debug: [2024-04-09T15:37:32.361209] __file__ = /home/mx/.cache/ruyi/progcache/0.8.0/__main__.py, sys.executable =                          __main__.py:47
                                    /home/mx/.cache/ruyi/progcache/0.8.0/python3, __compiled__ = __nuitka_version__(major=2, minor=1,                    
                                    micro=4, releaselevel='release', containing_dir='/usr/bin', standalone=True, onefile=True,                           
                                    macos_bundle_mode=False, no_asserts=False, no_docstrings=False, no_annotations=False, module=False)                  
debug: [2024-04-09T15:37:32.362643] argv[0] = ruyi, self_exe = /usr/bin/ruyi                                                               __main__.py:54
debug: [2024-04-09T15:37:32.386085] got defaults from system libcrypto libcrypto.so                                                      ssl_patch.py:113
debug: [2024-04-09T15:37:32.386784] X509_get_default_cert_file_env() = SSL_CERT_FILE                                                     ssl_patch.py:114
debug: [2024-04-09T15:37:32.387306] X509_get_default_cert_file() = /usr/lib/ssl/cert.pem                                                 ssl_patch.py:115
debug: [2024-04-09T15:37:32.387856] X509_get_default_cert_dir_env() = SSL_CERT_DIR                                                       ssl_patch.py:116
debug: [2024-04-09T15:37:32.388509] X509_get_default_cert_dir() = /usr/lib/ssl/certs                                                     ssl_patch.py:117
debug: [2024-04-09T15:37:32.473073] PyPI-style version of ruyi: 0.8.0                                                                       version.py:36
debug: [2024-04-09T15:37:32.473798] epoch 0 pre None post None                                                                              version.py:19
debug: [2024-04-09T15:37:32.477866] args=Namespace(func=<compiled_function cli_device_provision at 0x733083e5c640>)                       __init__.py:295

RuyiSDK Device Provisioning Wizard

This is a wizard intended to help you install a system on your device for your
development pleasure, all with ease.

You will be asked some questions that help RuyiSDK understand your device and
your intended configuration, then packages will be downloaded and flashed onto
the device's storage, that you should somehow make available on this host
system beforehand.

Note that, as Ruyi does not run as root, but raw disk access is most likely
required to flash images, you should arrange to allow your user account sudo
access to necessary commands such as dd. Flashing will fail if the sudo
configuration does not allow so.

Continue? (y/N) y

The following devices are currently supported by the wizard. Please pick your device:

  1. Allwinner Nezha D1
  2. Canaan Kendryte K230
  3. Milk-V Duo
  4. Milk-V Duo S
  5. Milk-V Mars
  6. Milk-V Mars CM
  7. Milk-V Meles
  8. Milk-V Pioneer Box
  9. Milk-V Vega
  10. SiFive HiFive Unmatched
  11. Sipeed Lichee RV
  12. Sipeed LicheePi 4A
  13. StarFive VisionFive
  14. StarFive VisionFive2

Choice? (1-14) 7

The device has the following variants. Please choose the one corresponding to your hardware at hand:

  1. Milk-V Meles (4G RAM)
  2. Milk-V Meles (8G RAM)

Choice? (1-2) 1

The following system configurations are supported by the device variant you have chosen. Please pick the one you want to put on the device:

  1. RevyOS for Milk-V Meles (4G RAM)

Choice? (1-1) 1
debug: [2024-04-09T15:37:35.101989] provisioning device variant 'milkv-meles@4g'                                                     provision_cli.py:121

We are about to download and install the following packages for your device:

 * board-image/revyos-milkv-meles
 * board-image/uboot-revyos-milkv-meles-4g

Proceed? (y/N) y
debug: [2024-04-09T15:37:35.942642] about to install for host linux/x86_64: {'board-image/revyos-milkv-meles',                             pkg_cli.py:220
                                    'board-image/uboot-revyos-milkv-meles-4g'}                                                                           
debug: [2024-04-09T15:37:35.952232] checking /home/mx/.cache/ruyi/distfiles/boot-milkv-meles-1.0.0.ext4                                    distfile.py:44
debug: [2024-04-09T15:37:35.952863] file /home/mx/.cache/ruyi/distfiles/boot-milkv-meles-1.0.0.ext4 not existent                           distfile.py:48
info: downloading https://github.com/milkv-meles/meles-images/releases/download/v1.0.0/boot.ext4 to 
/home/mx/.cache/ruyi/distfiles/boot-milkv-meles-1.0.0.ext4
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0
100  100M  100  100M    0     0  7104k      0  0:00:14  0:00:14 --:--:-- 13.3M
info: extracting boot-milkv-meles-1.0.0.ext4 for package revyos-milkv-meles-1.0.0
debug: [2024-04-09T15:37:50.761703] checking /home/mx/.cache/ruyi/distfiles/root-meles-20231210_134926.ext4.tar.gz                         distfile.py:44
debug: [2024-04-09T15:37:50.762475] file /home/mx/.cache/ruyi/distfiles/root-meles-20231210_134926.ext4.tar.gz not existent                distfile.py:48
info: downloading https://github.com/milkv-meles/meles-images/releases/download/v1.0.0/root-meles-20231210_134926.ext4.tar.gz to 
/home/mx/.cache/ruyi/distfiles/root-meles-20231210_134926.ext4.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0
100 1397M  100 1397M    0     0  25.4M      0  0:00:54  0:00:54 --:--:-- 32.2M
info: extracting root-meles-20231210_134926.ext4.tar.gz for package revyos-milkv-meles-1.0.0
debug: [2024-04-09T15:38:50.657875] about to call tar: argv=['tar', '-x', '-z', '-f',                                                       unpack.py:144
                                    '/home/mx/.cache/ruyi/distfiles/root-meles-20231210_134926.ext4.tar.gz', '--strip-components=1', '-C',               
                                    '/home/mx/.local/share/ruyi/blobs/.revyos-milkv-meles-1.0.0.tmpgwk_55y6']                                            
Traceback (most recent call last):
  File "/home/mx/.cache/ruyi/progcache/0.8.0/__main__.py", line 57, in <module>
  File "/home/mx/.cache/ruyi/progcache/0.8.0/ruyi/cli/__init__.py", line 300, in main
  File "/home/mx/.cache/ruyi/progcache/0.8.0/ruyi/device/provision_cli.py", line 30, in cli_device_provision
  File "/home/mx/.cache/ruyi/progcache/0.8.0/ruyi/device/provision_cli.py", line 110, in do_provision_interactive
  File "/home/mx/.cache/ruyi/progcache/0.8.0/ruyi/device/provision_cli.py", line 149, in do_provision_combo_interactive
  File "/home/mx/.cache/ruyi/progcache/0.8.0/ruyi/ruyipkg/pkg_cli.py", line 244, in do_install_atoms
  File "/home/mx/.cache/ruyi/progcache/0.8.0/ruyi/ruyipkg/pkg_cli.py", line 365, in do_install_blob_pkg
  File "/home/mx/.cache/ruyi/progcache/0.8.0/ruyi/ruyipkg/distfile.py", line 104, in unpack_or_symlink
  File "/home/mx/.cache/ruyi/progcache/0.8.0/ruyi/ruyipkg/unpack.py", line 80, in do_unpack_or_symlink
  File "/home/mx/.cache/ruyi/progcache/0.8.0/ruyi/ruyipkg/unpack.py", line 30, in do_unpack
OSError: [Errno 39] Directory not empty: '/home/mx/.local/share/ruyi/blobs/.revyos-milkv-meles-1.0.0.tmpgwk_55y6' -> '/home/mx/.local/share/ruyi/blobs/revyos-milkv-meles-1.0.0'

PS: The image is okay when I tried to manually unpack using tar xvf.

Reproduced on: Ubuntu 22.04.4 LTS & Arch Linux x86_64

weilinfox commented 3 months ago

貌似多个文件的镜像都会有类似的问题

$ RUYI_DEBUG=x ./ruyi device provision
debug: [2024-04-09T16:02:35.647820] __file__ = /home/hachi/.cache/ruyi/progcache/0.8.0/__main__.py, sys.executable = /home/hachi/.cache/ruyi/progcache/0.8.0/python3, __compiled__ = __nuitka_version__(major=2, minor=1,     __main__.py:47
                                    micro=4, releaselevel='release', containing_dir='/home/hachi/ruyisdk/ruyi-mugen', standalone=True, onefile=True, macos_bundle_mode=False, no_asserts=False, no_docstrings=False,                        
                                    no_annotations=False, module=False)                                                                                                                                                                     
debug: [2024-04-09T16:02:35.649046] argv[0] = ./ruyi, self_exe = /home/hachi/ruyisdk/ruyi-mugen/ruyi                                                                                                                          __main__.py:54
debug: [2024-04-09T16:02:35.667331] got defaults from system libcrypto libcrypto.so                                                                                                                                         ssl_patch.py:113
debug: [2024-04-09T16:02:35.667880] X509_get_default_cert_file_env() = SSL_CERT_FILE                                                                                                                                        ssl_patch.py:114
debug: [2024-04-09T16:02:35.668402] X509_get_default_cert_file() = /etc/ssl/cert.pem                                                                                                                                        ssl_patch.py:115
debug: [2024-04-09T16:02:35.668953] X509_get_default_cert_dir_env() = SSL_CERT_DIR                                                                                                                                          ssl_patch.py:116
debug: [2024-04-09T16:02:35.669447] X509_get_default_cert_dir() = /etc/ssl/certs                                                                                                                                            ssl_patch.py:117
debug: [2024-04-09T16:02:35.669993] get_default_verify_paths() values differ between bundled and system libssl                                                                                                               ssl_patch.py:45
debug: [2024-04-09T16:02:35.670610] bundled: DefaultVerifyPaths(cafile=None, capath=None, openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/usr/lib/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR',                     ssl_patch.py:48
                                    openssl_capath='/usr/lib/ssl/certs')                                                                                                                                                                    
debug: [2024-04-09T16:02:35.671377]  system: DefaultVerifyPaths(cafile='/etc/ssl/cert.pem', capath='/etc/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/etc/ssl/cert.pem',                                 ssl_patch.py:49
                                    openssl_capath_env='SSL_CERT_DIR', openssl_capath='/etc/ssl/certs')                                                                                                                                     
debug: [2024-04-09T16:02:35.748763] PyPI-style version of ruyi: 0.8.0                                                                                                                                                          version.py:36
debug: [2024-04-09T16:02:35.749408] epoch 0 pre None post None                                                                                                                                                                 version.py:19
debug: [2024-04-09T16:02:35.753535] args=Namespace(func=<compiled_function cli_device_provision at 0x7f65c1d64740>)                                                                                                          __init__.py:295

RuyiSDK Device Provisioning Wizard

This is a wizard intended to help you install a system on your device for your
development pleasure, all with ease.

You will be asked some questions that help RuyiSDK understand your device and
your intended configuration, then packages will be downloaded and flashed onto
the device's storage, that you should somehow make available on this host
system beforehand.

Note that, as Ruyi does not run as root, but raw disk access is most likely
required to flash images, you should arrange to allow your user account sudo
access to necessary commands such as dd. Flashing will fail if the sudo
configuration does not allow so.

Continue? (y/N) y

The following devices are currently supported by the wizard. Please pick your device:

  1. Allwinner Nezha D1
  2. Canaan Kendryte K230
  3. Milk-V Duo
  4. Milk-V Duo S
  5. Milk-V Mars
  6. Milk-V Mars CM
  7. Milk-V Meles
  8. Milk-V Pioneer Box
  9. Milk-V Vega
  10. SiFive HiFive Unmatched
  11. Sipeed Lichee RV
  12. Sipeed LicheePi 4A
  13. StarFive VisionFive
  14. StarFive VisionFive2

Choice? (1-14) 12

The device has the following variants. Please choose the one corresponding to your hardware at hand:

  1. Sipeed LicheePi 4A (8G RAM)
  2. Sipeed LicheePi 4A (16G RAM)

Choice? (1-2) 1

The following system configurations are supported by the device variant you have chosen. Please pick the one you want to put on the device:

  1. openEuler RISC-V (headless) for Sipeed LicheePi 4A (8G RAM)
  2. openEuler RISC-V (XFCE) for Sipeed LicheePi 4A (8G RAM)
  3. RevyOS for Sipeed LicheePi 4A (8G RAM)

Choice? (1-3) 1
debug: [2024-04-09T16:02:42.216698] provisioning device variant 'sipeed-lpi4a@8g'                                                                                                                                       provision_cli.py:121

We are about to download and install the following packages for your device:

 * board-image/oerv-sipeed-lpi4a-headless
 * board-image/uboot-oerv-sipeed-lpi4a-8g

Proceed? (y/N) y
debug: [2024-04-09T16:02:43.764495] about to install for host linux/x86_64: {'board-image/oerv-sipeed-lpi4a-headless', 'board-image/uboot-oerv-sipeed-lpi4a-8g'}                                                              pkg_cli.py:220
debug: [2024-04-09T16:02:43.783696] checking /home/hachi/.cache/ruyi/distfiles/boot-20231130-224203.ext4.zst                                                                                                                  distfile.py:44
debug: [2024-04-09T16:02:43.784871] file /home/hachi/.cache/ruyi/distfiles/boot-20231130-224203.ext4.zst not existent                                                                                                         distfile.py:48
info: downloading https://mirror.iscas.ac.cn/openeuler-sig-riscv/openEuler-RISC-V/preview/openEuler-23.09-V1-riscv64/lpi4a/boot-20231130-224203.ext4.zst to /home/hachi/.cache/ruyi/distfiles/boot-20231130-224203.ext4.zst
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  243M  100  243M    0     0  8416k      0  0:00:29  0:00:29 --:--:-- 8637k
info: extracting boot-20231130-224203.ext4.zst for package oerv-sipeed-lpi4a-headless-0.2309.1
debug: [2024-04-09T16:03:13.868316] about to call zstd: argv=['zstd', '-d', '/home/hachi/.cache/ruyi/distfiles/boot-20231130-224203.ext4.zst', '-o', './boot-20231130-224203.ext4']                                            unpack.py:231
/home/hachi/.cache/ruyi/distfiles/boot-20231130-224203.ext4.zst: 524288000 bytes 
debug: [2024-04-09T16:03:14.163059] checking /home/hachi/.cache/ruyi/distfiles/root-20231130-224203.ext4.zst                                                                                                                  distfile.py:44
debug: [2024-04-09T16:03:14.163699] file /home/hachi/.cache/ruyi/distfiles/root-20231130-224203.ext4.zst not existent                                                                                                         distfile.py:48
info: downloading https://mirror.iscas.ac.cn/openeuler-sig-riscv/openEuler-RISC-V/preview/openEuler-23.09-V1-riscv64/lpi4a/root-20231130-224203.ext4.zst to /home/hachi/.cache/ruyi/distfiles/root-20231130-224203.ext4.zst
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  905M  100  905M    0     0  8682k      0  0:01:46  0:01:46 --:--:-- 9760k
info: extracting root-20231130-224203.ext4.zst for package oerv-sipeed-lpi4a-headless-0.2309.1
debug: [2024-04-09T16:05:02.605330] about to call zstd: argv=['zstd', '-d', '/home/hachi/.cache/ruyi/distfiles/root-20231130-224203.ext4.zst', '-o', './root-20231130-224203.ext4']                                            unpack.py:231
/home/hachi/.cache/ruyi/distfiles/root-20231130-224203.ext4.zst: 4294967296 bytes 
Traceback (most recent call last):
  File "/home/hachi/.cache/ruyi/progcache/0.8.0/__main__.py", line 57, in <module>
  File "/home/hachi/.cache/ruyi/progcache/0.8.0/ruyi/cli/__init__.py", line 300, in main
  File "/home/hachi/.cache/ruyi/progcache/0.8.0/ruyi/device/provision_cli.py", line 30, in cli_device_provision
  File "/home/hachi/.cache/ruyi/progcache/0.8.0/ruyi/device/provision_cli.py", line 110, in do_provision_interactive
  File "/home/hachi/.cache/ruyi/progcache/0.8.0/ruyi/device/provision_cli.py", line 149, in do_provision_combo_interactive
  File "/home/hachi/.cache/ruyi/progcache/0.8.0/ruyi/ruyipkg/pkg_cli.py", line 244, in do_install_atoms
  File "/home/hachi/.cache/ruyi/progcache/0.8.0/ruyi/ruyipkg/pkg_cli.py", line 365, in do_install_blob_pkg
  File "/home/hachi/.cache/ruyi/progcache/0.8.0/ruyi/ruyipkg/distfile.py", line 104, in unpack_or_symlink
  File "/home/hachi/.cache/ruyi/progcache/0.8.0/ruyi/ruyipkg/unpack.py", line 80, in do_unpack_or_symlink
  File "/home/hachi/.cache/ruyi/progcache/0.8.0/ruyi/ruyipkg/unpack.py", line 30, in do_unpack
OSError: [Errno 39] Directory not empty: '/home/hachi/.local/share/ruyi/blobs/.oerv-sipeed-lpi4a-headless-0.2309.1.tmp0ip1ivbg' -> '/home/hachi/.local/share/ruyi/blobs/oerv-sipeed-lpi4a-headless-0.2309.1'

https://github.com/ruyisdk/ruyi/blob/24a701c18375eb4c8f59ecfb220b036a203e8253/ruyi/ruyipkg/unpack.py#L30C9-L30C38

这里 os.replace 会坏掉