openSUSE / zypper

World's most powerful command line package manager
http://en.opensuse.org/Portal:Zypper
Other
405 stars 110 forks source link

zypper enhance disk/download space checks #245

Open paigeadelethompson opened 5 years ago

paigeadelethompson commented 5 years ago

zypper dup anticipates final diskspace usage, as well as the amount of diskspace available but attempts to download more packages than it has disk space available for. It doesn't warn you of that or anything pre-flight it just presents you some facts about how much disk will be used how much total, etc. Wouldn't it be better if there were a failure mode or some kind of indication as to whether or not the operation can complete ?

Retrieving: apt-cacher-ng-3.1-1.3.x86_64.rpm ................................................................................................[error (363 B/s)]
Download (curl) error for 'http://download.opensuse.org/repositories/openSUSE:/Tumbleweed/standard/x86_64/apt-cacher-ng-3.1-1.3.x86_64.rpm':
Error code: Write error
Error message: Failed writing body (1108 != 15560)

Abort, retry, ignore? [a/r/i/...? shows all options] (a): 

➜  Resume git:(master) df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        3.5G  8.0K  3.5G   1% /dev
tmpfs           3.7G   99M  3.6G   3% /dev/shm
tmpfs           3.7G   19M  3.6G   1% /run
tmpfs           3.7G     0  3.7G   0% /sys/fs/cgroup
/dev/nvme0n1p3   40G   40G     0 100% /
/dev/nvme0n1p1  500M  164M  337M  33% /boot/efi
/dev/nvme0n1p3   40G   40G     0 100% /.snapshots
/dev/nvme0n1p3   40G   40G     0 100% /usr/local
/dev/nvme0n1p5  126G  116G  9.6G  93% /home
/dev/nvme0n1p3   40G   40G     0 100% /boot/grub2/x86_64-efi
/dev/nvme0n1p3   40G   40G     0 100% /tmp
/dev/nvme0n1p3   40G   40G     0 100% /opt
/dev/nvme0n1p3   40G   40G     0 100% /root
/dev/nvme0n1p3   40G   40G     0 100% /srv
/dev/nvme0n1p3   40G   40G     0 100% /var
/dev/nvme0n1p3   40G   40G     0 100% /boot/grub2/i386-pc
tmpfs           739M   44K  739M   1% /run/user/1000
➜  Resume git:(master) 

I will later attempt to remediate by relocating the newly downloaded RPMS to my home dir. Just thought I might throw this out there though I honestly couldn't even figure out why I'm being forced to update to a different version (15.1) suffice to say I'm a little bit annoyed at this point. I did check around a bit to see if there was any obvious reason (turned off all of my 3rd party repos, etc) for being forced to upgrade from 15.1 but just couldn't find it, fwiw. This operation became imparitive anyway after I installed a package (as I normally would when I need something) and several shared libraries were replaced causing missing linked symbols errors in some programs, also far from an ideal experience

paigeadelethompson commented 5 years ago

I don't mean to sound like I'm complaining just to be annoying, but I recently just switched to using SuSE after not using it since 2002 (I've been using Gentoo pretty consistently since then) and I kinda just accept the surprises from Eris as a given with portage, I was just kinda hoping for seamless to be a little more seemless. If I have the wrong idea about this I might as well just go back..

mlandres commented 5 years ago

I honestly couldn't even figure out why I'm being forced to update to a different version (15.1) ...

dup operates based on the set of enabled repositories. If I got you right and you had a Leap-15.1 installed, you should not use the openSUSE:/Tumbleweed repo. It's likely that dup switches to newer package versions it finds in TW. It's usually not a good idea to mix packages from TW and Leap. If this is actually the case, maybe this comment helps to check and repair it.

paigeadelethompson commented 5 years ago

Gotcha, yeah that would explain it though I'm not sure how I ended up with tumbleweed packages if I have leap installed, I must have added packages from tumbleweed by mistake thinking it was the same thing I had installed, the names leap and tumbleweed weren't entirely distinct to me until a minute ago :)

I don't mind terribly, though I'll need to be careful not to run into this problem on my server / hypervisor xD

also I just finished recompressing everything, and I'm gonna give this another shot with /var/cache/zypp linked to a dir on /home for now. Anyway i have a few things that may be worth noting about disk space usage:

➜  ~ df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        3.6G     0  3.6G   0% /dev
tmpfs           3.7G   60M  3.6G   2% /dev/shm
tmpfs           3.7G   11M  3.6G   1% /run
tmpfs           3.7G     0  3.7G   0% /sys/fs/cgroup
/dev/nvme0n1p3   40G   31G  8.9G  78% /
/dev/nvme0n1p3   40G   31G  8.9G  78% /.snapshots
/dev/nvme0n1p3   40G   31G  8.9G  78% /boot/grub2/i386-pc
/dev/nvme0n1p3   40G   31G  8.9G  78% /usr/local
/dev/nvme0n1p3   40G   31G  8.9G  78% /tmp
/dev/nvme0n1p1  500M  164M  337M  33% /boot/efi
/dev/nvme0n1p3   40G   31G  8.9G  78% /boot/grub2/x86_64-efi
/dev/nvme0n1p3   40G   31G  8.9G  78% /var
/dev/nvme0n1p3   40G   31G  8.9G  78% /opt
/dev/nvme0n1p3   40G   31G  8.9G  78% /srv
/dev/nvme0n1p3   40G   31G  8.9G  78% /root
/dev/nvme0n1p5  126G   80G   46G  64% /home
tmpfs           739M   32K  739M   1% /run/user/1000
➜  ~ 

IIRC df is not the best to use for btrfs:

linux-x9vc:/home/erratic # btrfs fi df /
Data, single: total=34.96GiB, used=28.33GiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=2.75GiB, used=1.71GiB
GlobalReserve, single: total=134.69MiB, used=0.00B
linux-x9vc:/home/erratic # btrfs fi df /home
Data, single: total=123.58GiB, used=78.22GiB
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=2.01GiB, used=1.17GiB
GlobalReserve, single: total=153.62MiB, used=0.00B
linux-x9vc:/home/erratic # mount | grep zlib
/dev/nvme0n1p3 on / type btrfs (rw,relatime,compress=zlib:3,ssd,space_cache,subvolid=267,subvol=/@/.snapshots/1/snapshot)
/dev/nvme0n1p3 on /.snapshots type btrfs (rw,relatime,compress=zlib:3,ssd,space_cache,subvolid=266,subvol=/@/.snapshots)
/dev/nvme0n1p3 on /boot/grub2/i386-pc type btrfs (rw,relatime,compress=zlib:3,ssd,space_cache,subvolid=265,subvol=/@/boot/grub2/i386-pc)
/dev/nvme0n1p3 on /usr/local type btrfs (rw,relatime,compress=zlib:3,ssd,space_cache,subvolid=259,subvol=/@/usr/local)
/dev/nvme0n1p3 on /tmp type btrfs (rw,relatime,compress=zlib:3,ssd,space_cache,subvolid=260,subvol=/@/tmp)
/dev/nvme0n1p3 on /boot/grub2/x86_64-efi type btrfs (rw,relatime,compress=zlib:3,ssd,space_cache,subvolid=264,subvol=/@/boot/grub2/x86_64-efi)
/dev/nvme0n1p3 on /var type btrfs (rw,relatime,compress=zlib:3,ssd,space_cache,subvolid=258,subvol=/@/var)
/dev/nvme0n1p3 on /opt type btrfs (rw,relatime,compress=zlib:3,ssd,space_cache,subvolid=263,subvol=/@/opt)
/dev/nvme0n1p3 on /srv type btrfs (rw,relatime,compress=zlib:3,ssd,space_cache,subvolid=261,subvol=/@/srv)
/dev/nvme0n1p3 on /root type btrfs (rw,relatime,compress=zlib:3,ssd,space_cache,subvolid=262,subvol=/@/root)
/dev/nvme0n1p5 on /home type btrfs (rw,relatime,compress=zlib:3,ssd,space_cache,subvolid=5,subvol=/)
linux-x9vc:/home/erratic # 

And I did a defrag / compress pass on both filesystems with btrfs fi defrag -r -czlib Now I've restarted zypper dup

12952 packages to upgrade, 1 to downgrade, 831 new, 129 to remove, 31  to change vendor, 10 to change arch.
Overall download size: 15.04 GiB. Already cached: 0 B. After the operation, additional 6.0 GiB will be used.
Continue? [y/n/...? shows all options] (y): y
(Use arrows or pgUp/pgDown keys to scroll the text by lines or pages.)

It seems like it should be able to do this now, and when it's finished there should be 2GB free of the 8.9GB thats free as it stands. I guess we'll see in a minute :)

paigeadelethompson commented 5 years ago

Actually I just decided last second not to because it would take way too long to finish and just removed the Tumbleweed repo and as per the fix link that you suggested that was indeed among one of the repositories that shouldn't have been enabled along with "msvec" (Michal Svec?) I think there was something I wanted from that but I can't remember what it was exactly, I think something having to do with certificates because I was trying to get secureboot to work right with a custom kernel that I compiled.

That's interesting though that somebody did the exact same thing that I did. Thanks for the help with that :) I would be interested though sometime in seeing how this can handle that big of an upgrade, just not today though xD

mlandres commented 5 years ago

Related bsc#1124282