Syntax error when trying to extend bookworm arm64 full image

rosesyrett commented 11 months ago

I downloaded the 2023-10-10-raspios-bookworm-arm64-full image, and tried to extend it by approx 3 Gb. This is a bit more than I actually need but demonstrates the problem:

/scratch$ sudo sdm --extend --xmb 3000 2023-10-10-raspios-bookworm-arm64-full.img 
[sudo] password for rose: 
* Extend IMG by 3000MB (3.1GB, 2.9GiB)...
2991587328 bytes (3.0 GB, 2.8 GiB) copied, 6 s, 499 MB/s
3000+0 records in
3000+0 records out
3145728000 bytes (3.1 GB, 2.9 GiB) copied, 6.07954 s, 517 MB/s
/usr/local/bin/sdm: line 187: 23.9GB: syntax error: invalid arithmetic operator (error token is ".9GB")
* Mount IMG '2023-10-10-raspios-bookworm-arm64-full.img'
mount: /dev/loop20p2 mounted on /mnt/sdm.
mount: /dev/loop20p1 mounted on /mnt/sdm/boot.
* Enter IMG '2023-10-10-raspios-bookworm-arm64-full.img'
root@sdm:~#

What's the easiest way to fix it? Thanks

gitbls commented 11 months ago

I can't repro in the simple test case, so some additional information would be a great start:

What version of sdm (sudo sdm --version)
What hardware
what OS/version (contents of /etc/os-release would be helpful)
What file system type (e.g., ext4, btrfs, etc) is /scratch?
What version is parted (sudo parted --version)

Please also provide the output from sudo parted -sm 2023-10-10-raspios-bookworm-arm64-full.img print for the IMG immediately after the failure.

rosesyrett commented 11 months ago

Thanks for the quick response.

version: sdm V9.6 /etc/os-release contents:

PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jamm

I just did sudo mkdir scratch at the root level; here is the file system types I've got:

rose@rose-Precision-3560:/scratch$ df -T
Filesystem     Type  1K-blocks      Used Available Use% Mounted on
tmpfs          tmpfs   1610216      2800   1607416   1% /run
/dev/nvme0n1p2 ext4  490617784 233069988 232552316  51% /
tmpfs          tmpfs   8051072   1241196   6809876  16% /dev/shm
tmpfs          tmpfs      5120         4      5116   1% /run/lock
tmpfs          tmpfs   8051072         0   8051072   0% /run/qemu
/dev/nvme0n1p1 vfat     523248     77460    445788  15% /boot/efi
tmpfs          tmpfs   1610212     38280   1571932   3% /run/user/1000

rose@rose-Precision-3560:/scratch$ sudo parted --version
parted (GNU parted) 3.4
Copyright (C) 2021 Free Software Foundation, Inc.
Licence GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by <http://git.debian.org/?p=parted/parted.git;a=blob_plain;f=AUTHORS>.

rose@rose-Precision-3560:/scratch$ sudo parted -sm 2023-10-10-raspios-bookworm-arm64-full.img print
BYT;
/scratch/2023-10-10-raspios-bookworm-arm64-full.img:23.9GB:file:512:512:msdos::;
1:4194kB:541MB:537MB:fat32::lba;
2:541MB:14.5GB:13.9GB:ext4::;

Let me know what hardware you're interested in and I can give details. Thanks

gitbls commented 11 months ago

In Raiders of the Lost Ark, Harrison Ford falls into a room full of snakes and says "Snakes! Why did it have to be snakes?"

Mine is "Ubuntu! Why did it have to be Ubuntu?" 🤣

The output of parted is very interesting. The last line of the output 2:541MB:14.5GB:13.9GB:ext4::; has something I've never seen...the "GB" units in the output. The code in sdm assumes that it's always MB, which is the cause of this issue.

Is the IMG really 14GB in size? What does ls -l 2023-10-10-raspios-bookworm-arm64-full.img show?

What are you trying to accomplish? Are you using sdm only to extend the IMG, or are you going to customize it as well? And do you really need the root partition to be 14GB?

Late-breaking news: With your parted output, I was able to repro the problem on a very large IMG. I will look into correcting this for the update that I'm hoping to release later today.

In the meantime, if you start with a fresh, unextended IMG and don't extend it "too much", parted will display the partition sizes in MB rather than GB and you'll be able to proceed. If you really need your root partition to be that large you'll need the updated version.

rosesyrett commented 11 months ago

My eventual aim with sdm, is to build a raspios image with a few bits of software installed. I have a plugin I've written that does this; basically just installs a python venv with things like scikit-image, and other bits of software, and setting them up in the .bashrc. I've worked I need about 2GB of space for this.

rose@rose-Precision-3560:/scratch$ ls -l 2023-10-10-raspios-bookworm-arm64-full.img
-rw-r--r-- 1 root root 23890755584 Nov 29 13:28 2023-10-10-raspios-bookworm-arm64-full.img
rose@rose-Precision-3560:/scratch$ du -h
22G .

Trying the whole process again, I get the following:

rose@rose-Precision-3560:/scratch$ sudo cp ~/Downloads/2023-10-10-raspios-bookworm-arm64-full.img .
rose@rose-Precision-3560:/scratch$ ls
2023-10-10-raspios-bookworm-arm64-full.img
rose@rose-Precision-3560:/scratch$ du -h
11G .
rose@rose-Precision-3560:/scratch$ sudo sdm --extend --xmb 512 2023-10-10-raspios-bookworm-arm64-full.img 
* Extend IMG by 512MB (0.5GB, 0.5GiB)...
512+0 records in
512+0 records out
536870912 bytes (537 MB, 512 MiB) copied, 0.255374 s, 2.1 GB/s
/usr/local/bin/sdm: line 187: 15.0GB: syntax error: invalid arithmetic operator (error token is ".0GB")
* Mount IMG '2023-10-10-raspios-bookworm-arm64-full.img'
mount: /dev/loop20p2 mounted on /mnt/sdm.
mount: /dev/loop20p1 mounted on /mnt/sdm/boot.
* Enter IMG '2023-10-10-raspios-bookworm-arm64-full.img'
root@sdm:~# exit
logout
umount: /mnt/sdm/boot unmounted
umount: /mnt/sdm unmounted
rose@rose-Precision-3560:/scratch$ du -h 2023-10-10-raspios-bookworm-arm64-full.img 
12G 2023-10-10-raspios-bookworm-arm64-full.img

gitbls commented 11 months ago

It's a siimple fix if you want to move ahead without waiting for the updated version later today.

Edit (with sudo) /usr/local/sdm/sdm
Locate the line done < <(parted -sm $ldimg print)
Change it to be done < <(parted -sm $ldimg unit MB print)
Save the file and exit

I highly recommend starting with a fresh copy of the IMG for your next try.

Please let me know if this works for you.

gitbls commented 11 months ago

My eventual aim with sdm, is to build a raspios image with a few bits of software installed. I have a plugin I've written that does this; basically just installs a python venv with things like scikit-image, and other bits of software, and setting them up in the .bashrc. I've worked I need about 2GB of space for this.

rose@rose-Precision-3560:/scratch$ ls -l 2023-10-10-raspios-bookworm-arm64-full.img
-rw-r--r-- 1 root root 23890755584 Nov 29 13:28 2023-10-10-raspios-bookworm-arm64-full.img
rose@rose-Precision-3560:/scratch$ du -h
22G   .

Trying the whole process again, I get the following:

rose@rose-Precision-3560:/scratch$ sudo cp ~/Downloads/2023-10-10-raspios-bookworm-arm64-full.img .
rose@rose-Precision-3560:/scratch$ ls
2023-10-10-raspios-bookworm-arm64-full.img
rose@rose-Precision-3560:/scratch$ du -h
11G   .
rose@rose-Precision-3560:/scratch$ sudo sdm --extend --xmb 512 2023-10-10-raspios-bookworm-arm64-full.img 
* Extend IMG by 512MB (0.5GB, 0.5GiB)...
512+0 records in
512+0 records out
536870912 bytes (537 MB, 512 MiB) copied, 0.255374 s, 2.1 GB/s
/usr/local/bin/sdm: line 187: 15.0GB: syntax error: invalid arithmetic operator (error token is ".0GB")
* Mount IMG '2023-10-10-raspios-bookworm-arm64-full.img'
mount: /dev/loop20p2 mounted on /mnt/sdm.
mount: /dev/loop20p1 mounted on /mnt/sdm/boot.
* Enter IMG '2023-10-10-raspios-bookworm-arm64-full.img'
root@sdm:~# exit
logout
umount: /mnt/sdm/boot unmounted
umount: /mnt/sdm unmounted
rose@rose-Precision-3560:/scratch$ du -h 2023-10-10-raspios-bookworm-arm64-full.img 
12G   2023-10-10-raspios-bookworm-arm64-full.img

Apparently parted on that particular version of Ubuntu (no clue about other versions) uses unit GB by default. Make the edit that I suggested and LMK how it works, as that is my preferred simple fix. If that doesn't work, the more complex fix is to fully handle GB in the parted output. Thx.

rosesyrett commented 11 months ago

That does seem to work:

rose@rose-Precision-3560:/scratch$ sudo sdm --extend --xmb 2048 2023-10-10-raspios-bookworm-arm64-full.img
* Extend IMG by 2048MB (2.1GB, 2.0GiB)...
1663041536 bytes (1.7 GB, 1.5 GiB) copied, 1 s, 1.7 GB/s
2048+0 records in
2048+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 1.26401 s, 1.7 GB/s
* Resize partition 2 of '2023-10-10-raspios-bookworm-arm64-full.img' to 16060MB
* Remount IMG to resize the file system
* Mount IMG '2023-10-10-raspios-bookworm-arm64-full.img'
mount: /dev/loop20p2 mounted on /mnt/sdm.
mount: /dev/loop20p1 mounted on /mnt/sdm/boot.
* Resize the file system
% (Ignore 'on-line resizing required' message)
resize2fs 1.46.5 (30-Dec-2021)
Filesystem at /dev/loop20p2 is mounted on /mnt/sdm; online resizing required
old_desc_blocks = 1, new_desc_blocks = 1
The filesystem on /dev/loop20p2 is now 3920882 (4k) blocks long.

umount: /mnt/sdm/boot unmounted
umount: /mnt/sdm unmounted

Thanks!

gitbls commented 11 months ago

Great...thanks for confirming. This will be in the next release, I'll update this issue once it's released.

gitbls commented 11 months ago

This is fixed in sdm V10.0.

gitbls commented 10 months ago

Closing due to no further activity.

gitbls / sdm

Syntax error when trying to extend bookworm arm64 full image #142