search

curl / curl

A command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, GOPHERS, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, MQTT, POP3, POP3S, RTMP, RTMPS, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, TELNET, TFTP, WS and WSS. libcurl offers a myriad of powerful features
https://curl.se/
Other
35.9k stars 6.43k forks source link

Increased CPU Usage on 8.10.0 With Alpine #14902

Closed Watercycle closed 1 month ago

Watercycle commented 1 month ago

I did this

I recently updated curl+libcurl on my php 8.3 fpm alpine server image from 8.9.1 to 8.10.0 (via apk --update upgrade) and observed a 2-3x bump in CPU utilization with however libcurl+PHP+Laravel+HTTPS and friends interact with with one another.

screenshots 1. The solid line is "cpu usage" today after reverting. The dotted line is yesterday. Both days had similar usage patterns. ![image](https://github.com/user-attachments/assets/b01198d4-d38b-4c1d-ae8e-7fb7a91c07f5) 2. Upgrade command, which is how I quickly isolated it to be a curl problem (or some curl sub-feature) ![image](https://github.com/user-attachments/assets/f499ea07-66af-46e0-8337-9ba75ba6a4c7)

Apologies, but I haven't had a chance to dig into the "why" further, yet since performance issues like this can be tedious to isolate. I figure this issue might just help others catch and/or remediate this surprise more quickly.

I expected the following

Similar performance characteristics to 8.9.1.

curl/libcurl version

/var/www/html # curl -V
curl 8.10.0 (x86_64-alpine-linux-musl) libcurl/8.10.0 OpenSSL/3.3.2 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 c-ares/1.28.1 libidn2/2.3.7 libpsl/0.21.5 nghttp2/1.62.1
Release-Date: 2024-09-11
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP UnixSockets zstd

operating system

/var/www/html # uname -a
Linux 57673a5cdd48 6.8.0-40-generic #40~22.04.3-Ubuntu SMP PREEMPT_DYNAMIC Tue Jul 30 17:30:19 UTC 2 x86_64 Linux

(Alpine 3.20.3)

bagder commented 1 month ago

We need specifics. How exactly do you use curl/libcurl? Protocols, versions, servers, patterns.

icing commented 1 month ago

@Watercycle , there is #14898 where libcurl goes into a busy loop, occupying a cpu core without getting the up/downloads any further. The fix for that was merged via https://github.com/curl/curl/pull/14901.

If you can build a libcurl yourself and throw that into your app, that would be worth seeing. But given that you relay on a complete stack that might be tricky.

Watercycle commented 1 month ago

I wasn't able to (quickly) reproduce it locally. But, it seems very likely this was related to #14898, so closing this issue and will dig into this a little more if I see the same thing on 8.10.1+.

Thank you for digging into this so quickly! And, apologies for not having a minimal-reproducible-example - posted an issue first since I figured it would be a few days before I could fully pin it down.