search

Piwigo / piwigo-videojs

Videojs port for piwigo. Play your videos on your web gallery!
http://piwigo.org/ext/extension_view.php?eid=610
GNU General Public License v3.0
64 stars 45 forks source link

Lot of ressources (CPU/RAM/Disk) used (since upgrade to v14.1.0) #218

Open mitch0667 opened 8 months ago

mitch0667 commented 8 months ago

Hello,

Since I upgraded to v14.1.0, I noticed that my server is consuming lot of ressources. While digging, several instances of ffmpeg are launched in the background when I navigate to Piwigo album that contain video.

root@piwigo2-748f6788b-cnpph:/# ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
abc          778 92.6  4.3 871868 707900 ?       Rl   20:03  13:31 ffmpeg -nostdin -loglevel error -i /tmp/magick-GI2FQHPLIWwYzmAHmkuC4ozhW6Po
abc          833 91.4  4.1 836788 672836 ?       Rl   20:04  12:25 ffmpeg -nostdin -loglevel error -i /tmp/magick-WyuU3yc9O50_nd1Ecsk3nH-CS9Hi
abc          994 49.4  0.5 256836 91968 ?        Rl   20:15   1:32 ffmpeg -nostdin -loglevel error -i /tmp/magick-nUMoaKrXhvYwX_Sf5I-PUX11tglo

At some point, I even experienced server that runs Piwigo not responding anymore.

This is what I get in /tmp folder:

root@piwigo2-748f6788b-cnpph:/# ls /tmp -lah
total 2.5G
drwxrwxrwt 1 root root 4.0K Jan 14 20:53 .
drwxr-xr-x 1 root root 4.0K Jan 14 18:44 ..
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:24 magick-4ZnQPDmYDihmBeTnfztWfzsMJPiOBahk -> /app/www/public/././upload/2023/11/11/20231111122325-9cae62d9.mp4
-rw------- 1 abc  abc   35M Jan 14 18:44 magick-5IgbVO0rcYpSo_Ol-FQEG3-f_MlbJbFE
lrwxrwxrwx 1 abc  abc    65 Jan 14 18:42 magick-5Zu6E3kZ_J1C3hZ6gehDHvb-cfPoeIDF -> /app/www/public/././upload/2023/11/12/20231112001702-3bc131a9.mov
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:02 magick-8wpxJRSVInF9PiS26Q47xrUzCtCAhdkm -> /app/www/public/././upload/2023/11/11/20231111122325-9cae62d9.mp4
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:12 magick-EJ9Ag2v-TgQIwfJ38hx24yGZBjbNkpMo -> /app/www/public/././upload/2023/11/12/20231112001702-3bc131a9.mov
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:03 magick-GI2FQHPLIWwYzmAHmkuC4ozhW6PoDAHm -> /app/www/public/././upload/2023/11/11/20231111122325-9cae62d9.mp4
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:23 magick-HcAhlnOqtUzFEDMn5iiZ4nWeGAOHLNAk -> /app/www/public/././upload/2023/11/11/20231111122325-9cae62d9.mp4
-rw------- 1 abc  abc     0 Jan 14 20:51 magick-Ji2GJfpgDQHnBCO6bgfZSQe9izpIGIlj
-rw------- 1 abc  abc  575M Jan 14 20:41 magick-O0gel3YbqfbBZd9fg_xlpuD4LkjiPjCN
-rw------- 1 abc  abc   35M Jan 14 20:14 magick-Q1Nj_BGEGIVF7OWNexVNgceKruGbGoNo
-rw------- 1 abc  abc     0 Jan 14 20:53 magick-Shdg8hZlz-Hu57cHfVU2NdTDZBAfCIAl
lrwxrwxrwx 1 abc  abc    65 Jan 14 18:42 magick-Wrajfutv4aygxOmU5QprZVIYEYEEDBHA -> /app/www/public/././upload/2023/11/12/20231112001702-3bc131a9.mov
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:04 magick-WyuU3yc9O50_nd1Ecsk3nH-CS9Hipfbl -> /app/www/public/././upload/2023/11/11/20231111122325-9cae62d9.mp4
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:53 magick-XCNpua3h8kMQ4DgWZXdzT7QkbSMljLAl -> /app/www/public/././upload/2023/11/11/20231111122325-9cae62d9.mp4
-rw------- 1 abc  abc   35M Jan 14 18:45 magick-bItUP5G0YPhBkgj8OjBVfrb3YoMMcoGA
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:52 magick-e0yxo5AJHMbpiw3yrwcnkZd38JcMkDej -> /app/www/public/././upload/2023/11/11/20231111122325-9cae62d9.mp4
lrwxrwxrwx 1 abc  abc    65 Jan 14 18:42 magick-f38EJZcenZZpDXiTglpcfnbixfbOdlnE -> /app/www/public/././upload/2023/11/12/20231112001702-3bc131a9.mov
-rw------- 1 abc  abc   35M Jan 14 18:45 magick-fuL-Pzar-7vbLQWk_9zo-8IEbNFpdinE
-rw------- 1 abc  abc  575M Jan 14 20:37 magick-hQf47wygHIj6GUFuTwT2wPkB32fBpLhk
-rw------- 1 abc  abc   35M Jan 14 18:45 magick-jzhWpokYLdI6KghV1Tdp1XEcbKMbKFDF
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:51 magick-kI49v3c0cmSjAR4pDEpxcKMLqYKfohlj -> /app/www/public/././upload/2023/11/11/20231111122325-9cae62d9.mp4
-rw------- 1 abc  abc     0 Jan 14 20:52 magick-kwErNoO3h8lGrBB0HdPY7K9cA0AIKAej
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:15 magick-nUMoaKrXhvYwX_Sf5I-PUX11tglojDLK -> /app/www/public/././upload/2023/11/12/20231112001702-3bc131a9.mov
-rw------- 1 abc  abc   35M Jan 14 20:15 magick-nijhOhSeCwg3FpMX97ilqzi9CtBApmMo
-rw------- 1 abc  abc  575M Jan 14 20:15 magick-oqg-z7sSiPsxAwLGIJ2v9_xPYDDfeDkm
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:25 magick-qaN13DaYZmWisVu8coVSJjozf_oJbnCN -> /app/www/public/././upload/2023/11/11/20231111122325-9cae62d9.mp4
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:20 magick-r4cx1MyPbnv8tZfMxxcgc85E26mllLEj -> /app/www/public/././upload/2023/11/11/20231111122325-9cae62d9.mp4
-rw------- 1 abc  abc   35M Jan 14 18:45 magick-t6zlfC_jT1fMA3fuwhP1FeuJaTEedoDe
lrwxrwxrwx 1 abc  abc    65 Jan 14 18:41 magick-tf2gJt8g3ek5FymtnA26VMLx7kijMfFE -> /app/www/public/././upload/2023/11/12/20231112001702-3bc131a9.mov
-rw------- 1 abc  abc  575M Jan 14 20:36 magick-tjoGUMM0w8xPi9lvCms0ZhDg70AJbIAk
lrwxrwxrwx 1 abc  abc    65 Jan 14 20:11 magick-xZo2vkgU04p_42KsGDQFzbbyTAaNiAOo -> /app/www/public/././upload/2023/11/12/20231112001702-3bc131a9.mov
lrwxrwxrwx 1 abc  abc    65 Jan 14 18:42 magick-yPjCVcAwWLzEa22li1mhwS4qNbcNEBEe -> /app/www/public/././upload/2023/11/12/20231112001702-3bc131a9.mov

Do you know what is the purpose of these ffmepg tasks? Is there any way to avoid it?

Thanks!

mitch0667 commented 8 months ago

Note: it even killed my k8s cluster :S

┌────────────────────────────────────────────────────────────── Nodes(all)[3] ───────────────────────────────────────────────────────────────┐
│ NAME↑                    STATUS                 ROLE                        VERSION                              PODS AGE                  │
│ k8s-master               Ready                  control-plane               v1.28.2                                 8 120d                 │
│ um250                    NotReady               <none>                      v1.28.2                                17 120d                 │
│ um300                    NotReady               <none>                      v1.28.2                                 8 120d                 │
Events:                                                                                                                                       
  Type     Reason                   Age                 From             Message                                                              
  ----     ------                   ----                ----             -------                                                              
  Warning  SystemOOM                46m                 kubelet          System OOM encountered, victim process: php-fpm82, pid: 498675       
  Warning  SystemOOM                42m                 kubelet          System OOM encountered, victim process: php-fpm82, pid: 498539       
  Warning  FreeDiskSpaceFailed      42m                 kubelet          Failed to garbage collect required amount of images. Attempted to fre
e 8805430067 bytes, but only found 0 bytes eligible to free.                                                                                  
  Normal   NodeHasSufficientMemory  39m (x8 over 11d)   kubelet          Node um250 status is now: NodeHasSufficientMemory                    
  Normal   NodeHasNoDiskPressure    39m (x11 over 11d)  kubelet          Node um250 status is now: NodeHasNoDiskPressure                      
  Normal   NodeHasSufficientPID     39m (x8 over 11d)   kubelet          Node um250 status is now: NodeHasSufficientPID                       
  Warning  SystemOOM                39m                 kubelet          System OOM encountered, victim process: php-fpm82, pid: 498618       
  Normal   NodeNotReady             39m (x2 over 11h)   kubelet          Node um250 status is now: NodeNotReady                               
  Normal   NodeReady                39m (x7 over 11d)   kubelet          Node um250 status is now: NodeReady                                  
  Warning  FreeDiskSpaceFailed      37m                 kubelet          Failed to garbage collect required amount of images. Attempted to fre
e 9408688947 bytes, but only found 0 bytes eligible to free.                                                                                  
  Warning  ImageGCFailed            37m                 kubelet          Failed to garbage collect required amount of images. Attempted to fre
e 9408688947 bytes, but only found 0 bytes eligible to free.                                                                                  
  Warning  FreeDiskSpaceFailed      32m                 kubelet          Failed to garbage collect required amount of images. Attempted to fre
e 9408807731 bytes, but only found 0 bytes eligible to free.                                                                                  
  Warning  ImageGCFailed            32m                 kubelet          Failed to garbage collect required amount of images. Attempted to fre
e 9408807731 bytes, but only found 0 bytes eligible to free.                                                                                  
  Warning  FreeDiskSpaceFailed      27m                 kubelet          Failed to garbage collect required amount of images. Attempted to fre
e 9408897843 bytes, but only found 0 bytes eligible to free.                                                                                  
  Warning  FreeDiskSpaceFailed      22m                 kubelet          Failed to garbage collect required amount of images. Attempted to fre
e 9408946995 bytes, but only found 0 bytes eligible to free.                                                                                  
  Warning  SystemOOM                17m                 kubelet          System OOM encountered, victim process: php-fpm82, pid: 499747       
  Normal   NodeNotReady             16m (x8 over 11d)   node-controller  Node um250 status is now: NodeNotReady