pop-os / upgrade

Utility for upgrading Pop!_OS and its recovery partition to new releases.
GNU General Public License v3.0
95 stars 29 forks source link

pop-upgrade 100% CPU usage #297

Open chase opened 2 years ago

chase commented 2 years ago

Running Pop_OS! 22.04.

I have automatic updates disabled, automatic install disabled, and update notifications set to Weekly.

I noticed that my fans started going full speed and htop showed that pop-upgrade daemon was at 100% CPU. The process tree showed that there was an apt-get instance that was defunct. I let it sit for a few hours before using kill -9 to stop it.


Another symptom that may be related, running apt update shows that it is unable to reach apt.pop-os.org

Visiting apt.pop-os.org in Firefox does not seem to be an issue.

GeodeCotin commented 2 years ago

Same issue here!

AnishDe12020 commented 2 years ago

It started recently for me too (22.04), the fan noise in unbearable, hope it is fixed soon!

Janvanoorschot commented 2 years ago

Same here (22.04). 8 cores running full speed for more then 15 minutes. Its doing some SSL encrypted traffic with an amazonaws host. Upgrade... really?

Janvanoorschot commented 2 years ago

Killed it with a -9 and it reported:

Aug 15 10:25:12 mouse systemd[1]: pop-upgrade.service: Consumed 45min 27.651s CPU time.
miaoerwu commented 2 years ago

@mmstick @jackpot51 I have the same problem. pop-upgrade daemon makes a cpu run 100% and does not stop. This causes several problems (that I have found so far) as follows.

  1. the laptop fan runs at high speed to dissipate heat, making a loud noise
  2. the battery power drops rapidly, originally can use about 10 hours, now can only use about 2 and a half hours
  3. need to wait for a long time to exit the process when shutting down, or forced shutdown
  4. the os upgrade & recovery page in settings keeps loading without any result
Janvanoorschot commented 2 years ago

I managed to trace it back (IMHO) to me activating Settings/OS Upgrade& Recovery/OS Recovery/Update Recovery Partition.

miaoerwu commented 2 years ago

I managed to trace it back (IMHO) to me activating Settings/OS Upgrade& Recovery/OS Recovery/Update Recovery Partition.

Yes, the pop-upgrade daemon process comes up after a while after each visit here

miaoerwu commented 2 years ago

@mmstick @jackpot51 I have the same problem. pop-upgrade daemon makes a cpu run 100% and does not stop. This causes several problems (that I have found so far) as follows.

  1. the laptop fan runs at high speed to dissipate heat, making a loud noise
  2. the battery power drops rapidly, originally can use about 10 hours, now can only use about 2 and a half hours
  3. need to wait for a long time to exit the process when shutting down, or forced shutdown
  4. the os upgrade & recovery page in settings keeps loading without any result

This looks like a network issue. I tried to tune the network via proxy or vpn Or just switch from network A to network B and found that I don't have these problems. Could it be that for some reason the network is not connecting and eventually the pop-upgrage daemon process is blocking?

chronos000 commented 1 year ago

I also see this problem being related to a network problem.

I left my computer running over night and the following morning I noticed the fans going at full blast. pop-upgrade was running at 100% cpu.

I tried to google about the problem, and noticed my internet access was behaving very strangely. It was not completely cut, but most sites would take unusually long to load or to fail loading. It could have been related to DNS not resolving, as pings were working normally.

I could not debug further as I just restarted the wifi and the connection returned to normal. Unfortunately, pop-upgrade remained blocked and and consuming 100% CPU. I tried to open "OS Upgrade & Recovery" in the settings menu of Pop!_OS, but it was unresponsive (the screen was stuck indefinitely at the rotating circle waiting for something to happen).

Running "sudo apt update" returned errors on some of the ubuntu repositories; for example:

Hit:1 http://apt.pop-os.org/proprietary jammy InRelease                      
Hit:2 http://apt.pop-os.org/release jammy InRelease                          
Ign:3 http://ar.archive.ubuntu.com/ubuntu jammy InRelease                    
Ign:4 http://ar.archive.ubuntu.com/ubuntu jammy-security InRelease
Ign:5 http://ar.archive.ubuntu.com/ubuntu jammy-updates InRelease
Ign:6 http://ar.archive.ubuntu.com/ubuntu jammy-backports InRelease
Ign:3 http://ar.archive.ubuntu.com/ubuntu jammy InRelease
Ign:4 http://ar.archive.ubuntu.com/ubuntu jammy-security InRelease
Ign:5 http://ar.archive.ubuntu.com/ubuntu jammy-updates InRelease
Ign:6 http://ar.archive.ubuntu.com/ubuntu jammy-backports InRelease
Ign:3 http://ar.archive.ubuntu.com/ubuntu jammy InRelease
Ign:4 http://ar.archive.ubuntu.com/ubuntu jammy-security InRelease
Ign:5 http://ar.archive.ubuntu.com/ubuntu jammy-updates InRelease
Ign:6 http://ar.archive.ubuntu.com/ubuntu jammy-backports InRelease
Err:3 http://ar.archive.ubuntu.com/ubuntu jammy InRelease
  Could not connect to ar.archive.ubuntu.com:80 (200.236.31.4), connection timed out
Err:4 http://ar.archive.ubuntu.com/ubuntu jammy-security InRelease
  Unable to connect to ar.archive.ubuntu.com:http:
Err:5 http://ar.archive.ubuntu.com/ubuntu jammy-updates InRelease
  Unable to connect to ar.archive.ubuntu.com:http:
Err:6 http://ar.archive.ubuntu.com/ubuntu jammy-backports InRelease
  Unable to connect to ar.archive.ubuntu.com:http:
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
W: Failed to fetch http://ar.archive.ubuntu.com/ubuntu/dists/jammy/InRelease  Could not connect to ar.archive.ubuntu.com:80 (200.236.31.4), connection timed out
W: Failed to fetch http://ar.archive.ubuntu.com/ubuntu/dists/jammy-security/InRelease  Unable to connect to ar.archive.ubuntu.com:http:
W: Failed to fetch http://ar.archive.ubuntu.com/ubuntu/dists/jammy-updates/InRelease  Unable to connect to ar.archive.ubuntu.com:http:
W: Failed to fetch http://ar.archive.ubuntu.com/ubuntu/dists/jammy-backports/InRelease  Unable to connect to ar.archive.ubuntu.com:http:
W: Some index files failed to download. They have been ignored, or old ones used instead.

It looks like the Argentinean mirror for ubuntu got offline since a few hours for some reason. I edited /etc/apt/sources.list.d/system.sources to use the main ubuntu archive, and I was able to apt update normally.

I never get this kind of errors, and I always update via terminal. So I definitely suspect this is related to the blocking state of pop-upgrade

mmstick commented 1 year ago

Try using apt.pop-os.org instead. The default repository in Pop. It is backed by a CDN that has servers across the globe.

chronos000 commented 1 year ago

Try using apt.pop-os.org instead. The default repository in Pop. It is backed by a CDN that has servers across the globe.

Thank you for your suggestion @mmstick

I installed Pop!_OS 22.04 a couple of months ago, and never changed the repositories. PopOS's installer must have chosen Argentinean Ubuntu mirror based on localization during installation.

Are you sure it safe to change /etc/apt/sources.list.d/system.sources from ubuntu's repos to the ones at PopOS?

mmstick commented 1 year ago

The installer does not make this change and installs are performed offline. You were supposed to already be on apt.pop-os.org automatically after the update was made a while back.

# cat /etc/apt/sources.list.d/system.sources
X-Repolib-Name: Pop_OS System Sources
Enabled: yes
Types: deb deb-src
URIs: http://apt.pop-os.org/ubuntu/
Suites: jammy jammy-security jammy-updates jammy-backports
Components: main restricted universe multiverse
X-Repolib-Default-Mirror: http://us.archive.ubuntu.com/ubuntu/
chase commented 1 year ago

Try using apt.pop-os.org instead. The default repository in Pop. It is backed by a CDN that has servers across the globe.

For what it's worth, when I originally posted it, for one reason or another apt.pop-os.org was not reachable, but seemingly only apt had issues reaching it.

I think the 100% CPU usage might have to do with a retry mechanism for fetching updates or something else. Instead of sleeping for an amount of time, it might be entering a loop that never exits or sleeps.


I haven't encountered the issue for the past couple of months, but I don't know if that's because the code for this tool has changed or because a tool like apt which is a dependency has been updated.

mmstick commented 1 year ago

I'm thinking it's probably a retry mechanism too

benjaminboruff commented 1 year ago

This still happens from time to time. Killing it -9 works ... Odd, and a bit annoying, but killing it is easy. (pop!_os 22.04 LTS with automatic updates on Oryx Pro 6)

rvente commented 11 months ago

This happens for me as well about once a month. I have a server running pop, so I don't always know when the CPU is going all-out. Kinda annoying so I just ran

sudo systemctl disable pop-upgrade.service
sudo systemctl stop pop-upgrade.service

Not sure if this is a good idea or if the solution will work. The issue is so intermittent that it will be a while before I know.

ported-pw commented 9 months ago

For me this sometimes happens on my laptop after unsuspending and might also be related to adverse network conditions (mobile hotspot not connected anymore after unsuspend etc.).

daliborfilus commented 3 months ago

Just happened for me on my NAS, Pop!_OS 22.04 LTS. Rebooted for memory upgrade, the reboot hang on Stopping pop-upgrade daemon... then it was killed by systemd after 1m 30s. Booted up, pop-upgrade daemon consumes 100% CPU (one core) for more than an hour now. So I run pop-upgrade from terminal and it hangs (for more than half-hour now) and does not respond to sigint.

Then I've done apt update, apt upgrade -y, and it worked - even while this thing is happening. I'm going to kill it and disable the daemon, thanks @rvente.

EDIT: it doesn't die easily. There are many child processes, many of them are pop-upgrade release check. CPU is consumed by the root process.