search

cockpit-project / cockpit

Cockpit is a web-based graphical interface for servers.
http://www.cockpit-project.org/
GNU Lesser General Public License v2.1
11.23k stars 1.11k forks source link

Is it normal for cockpit to be this slow? (15-20 seconds to load the page "accounts" with 2 entries) #18524

Open ThatCoffeeGuy opened 1 year ago

ThatCoffeeGuy commented 1 year ago

Explain what happens

  1. Install fedora 37 with headless tools selected
  2. Navigate to copilot's page
  3. Check the speed

Opening network, CPU, logs. They all take really, really long to load sometimes. Waiting 20 seconds to open a page with 2 accounts listed there is strange.

My question is, is this normal behavior?

This is running on a Ryzen 5700X, 64GB fast RAM, NVME, etc. I really wouldn't expect it being this slow. It gets better, but after a few hours, it seems to be painfully slow again.

Version of Cockpit

287

Where is the problem in Cockpit?

Overview

Server operating system

Fedora

Server operating system version

Fedora Linux 37 (Server Edition) x86_64

What browsers are you using?

Chrome

System log

TLS errors are being spammed, but nothing else seems out of ordinary.

Mar 14 13:34:00 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:00 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:00 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:04 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:34:34 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:35:43 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:35:43 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:35:43 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:35:43 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:35:50 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:35:50 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Mar 14 13:36:20 sherper cockpit-tls[16213]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
martinpitt commented 1 year ago

No, it's not normal. After opening Cockpit, depending on your CPU and networking speed they can take 0.2 to 1 secon. But 20s is way to long. One or two TLS errors are normal if you connect to cockpit with a self-signed certificate, and the browser rejects the initial connection. That many suggests something else. At the moment there is no indication what could be the problem.

Is this all on your local machine, or are you connecting to a remote server? Do you perhaps have some browser extensions or firewalls, etc? Can you try this in a private window, or even with another browser (Firefox, GNOME Web)?

StuartIanNaylor commented 11 months ago

With Ubuntu 22.04 I can see in htop cockpit-tls is running near 100% on a single core. Linux ubuntu 6.6.0 #1 SMP PREEMPT aarch64 GNU/Linux

Architecture:           aarch64
  CPU op-mode(s):       32-bit, 64-bit
  Byte Order:           Little Endian
CPU(s):                 8
  On-line CPU(s) list:  0-7
Vendor ID:              ARM
  Model name:           Cortex-A55
    Model:              0
    Thread(s) per core: 1
    Core(s) per socket: 4
    Socket(s):          1
    Stepping:           r2p0
    CPU max MHz:        1800.0000
    CPU min MHz:        408.0000
    BogoMIPS:           48.00
    Flags:              fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp
  Model name:           Cortex-A76
    Model:              0
    Thread(s) per core: 1
    Core(s) per socket: 4
    Socket(s):          1
    Stepping:           r4p0
    CPU max MHz:        2208.0000
    CPU min MHz:        408.0000
    BogoMIPS:           48.00
    Flags:              fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp

Not sure why cockpit-tls is so heavy?

martinpitt commented 11 months ago

@StuartIanNaylor it's definitively not meant to be that heavy -- we even run CI on aarch64, and I also (sometimes) ran Cockpit on my old RasPi 3 even (which was much less powerful than yours). But with so little available information it needs some actual debugging on your system. It sounds like some GnuTLS operation is going crazy..

StuartIanNaylor commented 11 months ago

I don't get a lot in the journal than

Nov 14 13:46:33 ubuntu systemd[1]: Starting Socket for Cockpit Web Service http instance...
Nov 14 13:46:33 ubuntu systemd[1]: Starting Socket for Cockpit Web Service https instance factory...
Nov 14 13:46:33 ubuntu systemd[1]: Listening on Socket for Cockpit Web Service http instance.
Nov 14 13:46:33 ubuntu systemd[1]: Listening on Socket for Cockpit Web Service https instance factory.
Nov 14 13:46:33 ubuntu systemd[1]: Starting Cockpit Web Service...
Nov 14 13:46:33 ubuntu systemd[1]: Started Cockpit Web Service.
Nov 14 13:46:33 ubuntu cockpit-tls[4411]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:46:34 ubuntu cockpit-tls[4411]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:46:34 ubuntu cockpit-tls[4411]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:46:34 ubuntu systemd[1]: Created slice Slice /system/cockpit-wsinstance-https-factory.
Nov 14 13:46:34 ubuntu systemd[1]: Started Cockpit Web Service https instance factory (PID 4411/UID 113).
Nov 14 13:46:34 ubuntu systemd[1]: Created slice Slice /system/cockpit-wsinstance-https.
Nov 14 13:46:34 ubuntu systemd[1]: cgroup compatibility translation between legacy and unified hierarchy settings activated. See cgroup-compat debug messages for details.
Nov 14 13:46:34 ubuntu systemd[1]: Created slice Resource limits for all cockpit-ws-https@.service instances.
Nov 14 13:46:34 ubuntu systemd[1]: Starting Socket for Cockpit Web Service https instance e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855...
Nov 14 13:46:34 ubuntu systemd[1]: Listening on Socket for Cockpit Web Service https instance e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.
Nov 14 13:46:34 ubuntu systemd[1]: Started Cockpit Web Service https instance e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.
Nov 14 13:46:34 ubuntu systemd[1]: cockpit-wsinstance-https-factory@0-4411-113.service: Deactivated successfully.
Nov 14 13:46:34 ubuntu cockpit-tls[4411]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:46:34 ubuntu cockpit-tls[4411]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:46:35 ubuntu cockpit-tls[4411]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:46:48 ubuntu cockpit-session[4426]: pam_ssh_add: Failed adding some keys
Nov 14 13:46:48 ubuntu cockpit-session[4426]: pam_unix(cockpit:session): session opened for user ubuntu(uid=1000) by (uid=0)
Nov 14 13:46:48 ubuntu systemd-logind[1077]: New session 4 of user ubuntu.
Nov 14 13:46:48 ubuntu systemd[1]: Started Session 4 of User ubuntu.
Nov 14 13:46:49 ubuntu polkitd(authority=local)[1074]: Registered Authentication Agent for unix-session:4 (system bus name :1.29 [cockpit-bridge], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8)
Nov 14 13:46:49 ubuntu sudo[4438]:   ubuntu : PWD=/run/user/1000 ; USER=root ; COMMAND=/usr/bin/cockpit-bridge --privileged
Nov 14 13:46:49 ubuntu sudo[4438]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1000)
Nov 14 13:46:49 ubuntu cockpit-tls[4411]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:47:05 ubuntu polkitd(authority=local)[1074]: Unregistered Authentication Agent for unix-session:4 (system bus name :1.29, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8)
Nov 14 13:47:05 ubuntu sudo[4438]: pam_unix(sudo:session): session closed for user root
Nov 14 13:47:05 ubuntu cockpit-session[4426]: pam_unix(cockpit:session): session closed for user ubuntu
Nov 14 13:47:05 ubuntu systemd[1]: Starting Download data for packages that failed at package install time...
Nov 14 13:47:05 ubuntu systemd[1]: session-4.scope: Deactivated successfully.
Nov 14 13:47:05 ubuntu systemd-logind[1077]: Session 4 logged out. Waiting for processes to exit.
Nov 14 13:47:05 ubuntu systemd-logind[1077]: Removed session 4.
Nov 14 13:47:05 ubuntu systemd[1]: update-notifier-download.service: Deactivated successfully.
Nov 14 13:47:05 ubuntu systemd[1]: Finished Download data for packages that failed at package install time.
Nov 14 13:47:20 ubuntu cockpit-ws[4418]: request timed out, closing
Nov 14 13:47:20 ubuntu cockpit-tls[4411]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:47:20 ubuntu cockpit-tls[4411]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:48:20 ubuntu systemd[1]: cockpit-wsinstance-https@e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.service: Deactivated successfully.
Nov 14 13:48:20 ubuntu systemd[1]: cockpit-wsinstance-https@e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.socket: Deactivated successfully.
Nov 14 13:48:20 ubuntu systemd[1]: Closed Socket for Cockpit Web Service https instance e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.
Nov 14 13:49:20 ubuntu systemd[1]: cockpit.service: Deactivated successfully.
Nov 14 13:49:20 ubuntu systemd[1]: cockpit-wsinstance-http.socket: Deactivated successfully.
Nov 14 13:49:20 ubuntu systemd[1]: Closed Socket for Cockpit Web Service http instance.
Nov 14 13:49:20 ubuntu systemd[1]: cockpit-wsinstance-https-factory.socket: Deactivated successfully.
Nov 14 13:49:20 ubuntu systemd[1]: Closed Socket for Cockpit Web Service https instance factory.
Nov 14 13:50:58 ubuntu systemd[1]: Starting Daily apt upgrade and clean activities...
Nov 14 13:51:01 ubuntu systemd[1]: apt-daily-upgrade.service: Deactivated successfully.
Nov 14 13:51:01 ubuntu systemd[1]: Finished Daily apt upgrade and clean activities.
Nov 14 13:51:19 ubuntu systemd[1]: Starting Socket for Cockpit Web Service http instance...
Nov 14 13:51:19 ubuntu systemd[1]: Starting Socket for Cockpit Web Service https instance factory...
Nov 14 13:51:19 ubuntu systemd[1]: Listening on Socket for Cockpit Web Service http instance.
Nov 14 13:51:19 ubuntu systemd[1]: Listening on Socket for Cockpit Web Service https instance factory.
Nov 14 13:51:19 ubuntu systemd[1]: Starting Cockpit Web Service...
Nov 14 13:51:19 ubuntu systemd[1]: Started Cockpit Web Service.
Nov 14 13:51:19 ubuntu cockpit-tls[4576]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:51:19 ubuntu systemd[1]: Started Cockpit Web Service https instance factory (PID 4576/UID 113).
Nov 14 13:51:19 ubuntu systemd[1]: Starting Socket for Cockpit Web Service https instance e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855...
Nov 14 13:51:19 ubuntu systemd[1]: Listening on Socket for Cockpit Web Service https instance e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.
Nov 14 13:51:19 ubuntu systemd[1]: Started Cockpit Web Service https instance e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855.
Nov 14 13:51:19 ubuntu systemd[1]: cockpit-wsinstance-https-factory@1-4576-113.service: Deactivated successfully.
Nov 14 13:51:19 ubuntu cockpit-session[4584]: pam_ssh_add: Failed adding some keys
Nov 14 13:51:19 ubuntu cockpit-session[4584]: pam_unix(cockpit:session): session opened for user ubuntu(uid=1000) by (uid=0)
Nov 14 13:51:19 ubuntu systemd-logind[1077]: New session 5 of user ubuntu.
Nov 14 13:51:19 ubuntu systemd[1]: Started Session 5 of User ubuntu.
Nov 14 13:51:19 ubuntu polkitd(authority=local)[1074]: Registered Authentication Agent for unix-session:5 (system bus name :1.35 [cockpit-bridge], object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8)
Nov 14 13:51:20 ubuntu sudo[4597]:   ubuntu : PWD=/run/user/1000 ; USER=root ; COMMAND=/usr/bin/cockpit-bridge --privileged
Nov 14 13:51:20 ubuntu sudo[4597]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1000)
Nov 14 13:51:20 ubuntu cockpit-ws[4591]: invalid or unusable locale: en_GB.UTF-8
Nov 14 13:51:20 ubuntu cockpit-tls[4576]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:51:20 ubuntu cockpit-tls[4576]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:51:20 ubuntu cockpit-tls[4576]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:51:20 ubuntu cockpit-tls[4576]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:51:20 ubuntu cockpit-tls[4576]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:51:20 ubuntu dbus-daemon[1062]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.36' (uid=1000 pid=4591 comm="cockpit-bridge ")
Nov 14 13:51:20 ubuntu cockpit-tls[4576]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:51:20 ubuntu systemd[1]: Starting Hostname Service...
Nov 14 13:51:20 ubuntu cockpit-tls[4576]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:51:20 ubuntu cockpit-tls[4576]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:51:20 ubuntu cockpit-tls[4576]: cockpit-tls: gnutls_handshake failed: A TLS fatal alert has been received.
Nov 14 13:51:20 ubuntu dbus-daemon[1062]: [system] Successfully activated service 'org.freedesktop.hostname1'
Nov 14 13:51:20 ubuntu systemd[1]: Started Hostname Service.
Nov 14 13:51:20 ubuntu dbus-daemon[1062]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.40' (uid=0 pid=4598 comm="cockpit-bridge --privil>
Nov 14 13:51:20 ubuntu dbus-daemon[1062]: [system] Activating via systemd: service name='org.freedesktop.PackageKit' unit='packagekit.service' requested by ':1.43' (uid=0 pid=4598 comm="cockpit-bridge --privileged ")
Nov 14 13:51:20 ubuntu systemd[1]: Starting Time & Date Service...
Nov 14 13:51:20 ubuntu systemd[1]: Starting PackageKit Daemon...
Nov 14 13:51:20 ubuntu PackageKit[4628]: daemon start
Nov 14 13:51:20 ubuntu dbus-daemon[1062]: [system] Successfully activated service 'org.freedesktop.timedate1'
Nov 14 13:51:20 ubuntu systemd[1]: Started Time & Date Service.
Nov 14 13:51:20 ubuntu dbus-daemon[1062]: [system] Successfully activated service 'org.freedesktop.PackageKit'
Nov 14 13:51:20 ubuntu systemd[1]: Started PackageKit Daemon.
Nov 14 13:51:21 ubuntu PackageKit[4628]: resolve transaction /133_ebeedecd from uid 0 finished with success after 360ms
Nov 14 13:51:22 ubuntu PackageKit[4628]: get-updates transaction /134_abccbcab from uid 0 finished with success after 688ms

Thats the journal with 1 unsucessful followed by 1 sucess login and got in 2nd time that time. I have found that using Chrome you get errors using Edge which is based on Chrome no errors!? Chrome is up to date Version 119.0.6045.124 (Official Build) (64-bit) What is even stranger after logging in with Edge now Chromw works every time? Figure that?

v3tro86 commented 9 months ago

Ok so maybe this could be helpful:

i had 389 running on centos 9 vm as a test for a customer project i wanted to launch. They have rhel8/9 mostly so i said well ok need to test ok rocky 9 then too (scripts etc)

so i installed as on centos 9 dscreate etc and in cockpit it was running like up to 5 minutes load time for the tree in ldap every single entry whereas on centos it was like instantly there and everything was fast in cockpit.

Long story, short: i figured out you MUST use fqdn in dscreate and have it - at best - in /etc/hosts xx.xx.xx.xx fqdn too..

Since that on rocky 9 it also runs smooth as hell

Regards

link89 commented 8 months ago

I have a ubuntu workstation in office LAN and whenever I connect to cockpit it just took a very long time to connect via IP address. And sometimes it would just failed due to timeout.

And I have no idea why the cockpit-tls process use 200% of CPU

h1ght commented 5 months ago

1 SMP PREEMPT_DYNAMIC Debian 6.6.13-1~bpo12+1 (2024-02-15) on debian bookworm cockpit-tls eats alot of cores when connecting after restarts. kind of the same behaviour, takes "ages" to load a page. cpu is a intel i3-12100,

ernesqba commented 4 months ago

hi, I have the same problem. I'm using cockpit in a local server with ubuntu 24.04. When I installed cockpit was really great and quick but right now it is too slow. No idea what is happening.