search

jhuckaby / Cronicle

A simple, distributed task scheduler and runner with a web based UI.
http://cronicle.net
Other
3.89k stars 389 forks source link

php shell script and zombie processes #753

Open Giattuck opened 6 months ago

Giattuck commented 6 months ago

Summary

Hello, i moved around 50 php shell script from an old server to a new one with cronicle.

I run those scripts with the default shell plugin like this:

#!/bin/sh

php myPHPScript.php
echo "Cronjob completed."

All my script are working well!

But i have one script, that is the official magento shell that runs the magento's jobs:

#!/bin/sh

php /var/www/magento2/bin/magento cron:run
echo "Cronjob completed."

it runs every minutes and do the jobs, but i have new zombie php processes at every run:

    PID USER      PR  NI    VIRT    RES  %CPU  %MEM     TIME+ S COMMAND                                                                                                                     
   2001 root      20   0    0.0m   0.0m   0.0   0.0   0:00.96 Z  `- [php7] <defunct>                                                                                                        
   2003 root      20   0    0.0m   0.0m   0.0   0.0   0:00.98 Z  `- [php7] <defunct>                                                                                                        
   2034 root      20   0    0.0m   0.0m   0.0   0.0   0:01.07 Z  `- [php7] <defunct>                                                                                                        
   2036 root      20   0    0.0m   0.0m   0.0   0.0   0:00.99 Z  `- [php7] <defunct>                                                                                                        
   2038 root      20   0    0.0m   0.0m   0.0   0.0   0:00.97 Z  `- [php7] <defunct>                                                                                                        
   2040 root      20   0    0.0m   0.0m   0.0   0.0   0:00.97 Z  `- [php7] <defunct>

The questions are:

1) is it ok to runs my scripts like i did or there are some better soultions? (custom plugin?) NB: i prefer to not have to modify around 50 php scripts 2) how i can prevent that magento script make zombies? i cant remove them because the main job doesn't not exists anymore, i think i have to remove them inside the script before it ends, but i don't know how.

Any help?

Thank you and sorry for my english

Your Setup

i'm running cronicle inside a custom docker image made by me.

Operating system and version?

Alpine Linux

Node.js version?

16.20.2

Cronicle software version?

0.9.44

Are you using a multi-server setup, or just a single server?

no

Are you using the filesystem as back-end storage, or S3/Couchbase?

filesystem

Can you reproduce the crash consistently?

no crash. every run i have 4 zombie processes

Giattuck commented 6 months ago

I add some more info:

when the script is running:

   3886 root      20   0  255.8m  34.5m   0.7   0.9   0:00.27 S  `- node bin/shell-plugin.js                                                                                                
   3916 root      20   0  167.9m 116.8m  32.2   3.0   0:00.49 R  `- /usr/bin/php7 /var/www/magento2/bin/magento cron:run --group=index --bootstrap=standaloneProcessStarted=1               
   3918 root      20   0  183.1m 131.3m  36.2   3.4   0:00.55 S  `- /usr/bin/php7 /var/www/magento2/bin/magento cron:run --group=consumers --bootstrap=standaloneProcessStarted=1           
   3920 root      20   0  185.1m 131.5m  36.8   3.4   0:00.56 S  `- /usr/bin/php7 /var/www/magento2/bin/magento cron:run --group=amasty_base --bootstrap=standaloneProcessStarted=1         
   3922 root      20   0  175.9m 124.6m  32.9   3.2   0:00.50 R  `- /usr/bin/php7 /var/www/magento2/bin/magento cron:run --group=amasty_groupassign --bootstrap=standaloneProcessStarted=1

when the script ends:

   3916 root      20   0    0.0m   0.0m   0.0   0.0   0:01.07 Z  `- [php7] <defunct>                                                                                                        
   3918 root      20   0    0.0m   0.0m   0.0   0.0   0:00.98 Z  `- [php7] <defunct>                                                                                                        
   3920 root      20   0    0.0m   0.0m   0.0   0.0   0:00.97 Z  `- [php7] <defunct>                                                                                                        
   3922 root      20   0    0.0m   0.0m   0.0   0.0   0:00.96 Z  `- [php7] <defunct>
mikeTWC1984 commented 6 months ago

Not 100% if it will help but try to use tini tool (if it's not install do "apk add tini") tini -- php /var/www/magento2/bin/magento cron:run

Cronicle has similar issue when it's running in container, so that's the workaround

https://github.com/krallin/tini

Giattuck commented 6 months ago

thanks for the reply @mikeTWC1984 !!

I've tryed with tini -- php and i got this:

[WARN  tini (110)] Tini is not running as PID 1 and isn't registered as a child subreaper.
Zombie processes will not be re-parented to Tini, so zombie reaping won't work.
To fix the problem, use the -s option or set the environment variable TINI_SUBREAPER to register Tini as a child subreaper, or run Tini as PID 1.

so i tryed with the -s option and zombies are there :(

Giattuck commented 6 months ago

Update:

with this:

#!/bin/sh

tini -sgvvv -- php /var/www/magento2/bin/magento cron:run

first time worked:

[TRACE tini (933)] Registered as child subreaper
Ran jobs by schedule.
[INFO  tini (933)] Spawned child process 'php' with pid '934'
[TRACE tini (933)] No child to reap
[TRACE tini (933)] No child to reap
[TRACE tini (933)] No child to reap
[DEBUG tini (933)] Received SIGCHLD
[DEBUG tini (933)] Reaped child with pid: '947'
[TRACE tini (933)] No child to reap
[DEBUG tini (933)] Received SIGCHLD
[DEBUG tini (933)] Reaped child with pid: '949'
[TRACE tini (933)] No child to reap
[DEBUG tini (933)] Received SIGCHLD
[DEBUG tini (933)] Reaped child with pid: '945'
[TRACE tini (933)] No child to reap
[DEBUG tini (933)] Received SIGCHLD
[DEBUG tini (933)] Reaped child with pid: '943'
[TRACE tini (933)] No child to reap
[DEBUG tini (933)] Received SIGCHLD
[DEBUG tini (933)] Reaped child with pid: '951'
[TRACE tini (933)] No child to reap
[DEBUG tini (933)] Received SIGCHLD
[DEBUG tini (933)] Reaped child with pid: '934'
[INFO  tini (933)] Main child exited normally (with status '0')
[TRACE tini (933)] No child to wait
[TRACE tini (933)] Exiting: child has exited

and then doesn't work anymore:

[TRACE tini (1092)] Registered as child subreaper
Ran jobs by schedule.
[INFO  tini (1092)] Spawned child process 'php' with pid '1093'
[TRACE tini (1092)] No child to reap
[DEBUG tini (1092)] Received SIGCHLD
[DEBUG tini (1092)] Reaped child with pid: '1093'
[INFO  tini (1092)] Main child exited normally (with status '0')
[TRACE tini (1092)] No child to reap
[TRACE tini (1092)] Exiting: child has exited
jhuckaby commented 6 months ago

Try using the docker --init flag: https://docs.docker.com/config/containers/multi-service_container/

The --init flag inserts a tiny init-process into the container as the main process, and handles reaping of all processes when the container exits.

Giattuck commented 6 months ago

@jhuckaby solution worked! Thank you very much!!

I had to add init:true in my compose file. With this i have docker-init as pid 1, cronicle as pid 7 and no more zombies.

 PID USER      PR  NI    VIRT    RES  %CPU  %MEM     TIME+ S COMMAND                                                                                                                                                                                      
      1 root      20   0    0.8m   0.4m   0.0   0.0   0:00.24 S /sbin/docker-init -- node bin/docker-entrypoint.js                                                                                                                                           
      7 root      20   0  289.3m  68.1m   0.7   1.8   0:48.12 S  `- Cronicle Server