search

alexsilva / supervisor

Supervisor process control system for Windows
http://supervisord.org
Other
118 stars 26 forks source link

Error/Unexpected Behaviour while stopping service. Need help #24

Closed tariksetia closed 4 years ago

tariksetia commented 4 years ago

supervisor-win version: 4.4.0

I have created an executable(processor.exe) of a python multiprocessing program using pyinstaller. I am trying to deploy this exe as windows service. The problem is I am not able to stop the service.

following are the contents of supervisord.conf: 

[program:monitor]
command=C:/Users/monitor/Desktop/app/src/service/processor.exe
directory =  C:/Users/monitor/Desktop/app/src/service

[supervisord]
logfile = C:/Users/monitor/Desktop/app/src/service/svc/supervisord.log
logfile_maxbytes = 50MB
logfile_backups=10
loglevel = debug
pidfile = C:/Users/monitor/Desktop/app/src/service/svc/supervisord.pid
nodaemon = false
stopwaitsecs = 5
killasgroup = true
stopasgroup = true

[supervisorctl]

[inet_http_server]
port = 127.0.0.1:9001
username = admin
password = admin

There is no error is service.log file created by supervisor. I am not sure if I am doing something wrong. Any help would greatly appreciated.

Thank!

alexsilva commented 4 years ago

Can you install the version of github ? There are improvements to the service log. Done attach the logs after running the service and trying to stop it.

The service logs are in the same location as supervisor.conf or in the python directory (python home).

tariksetia commented 4 years ago

I used following command to install GitHub version:

python -m pip install git+https://github.com/alexsilva/supervisor.git@windows -U

Then I updated the service using python -m supervisor.service update -c <abs path to config file>

I got an error (in service.log) while stopping service via windows services and supervisor.severices stop command. But when I tried to stopped via web interface, it did nothing other than changing status to stopped on the webpage All the processes were still running.

following are the logs:


2020-08-19 16:44:34,478 [INFO] supervisor config path: C:\Users\monitor\Desktop\app\src\service\supervisord.conf
2020-08-19 16:44:34,711 [INFO] supervisor starting...
2020-08-19 16:45:20,904 [INFO] supervisorctl shutdown
2020-08-19 16:45:22,935 [INFO] Server requires authentication
2020-08-19 16:45:24,952 [INFO] error: <class 'xmlrpc.client.ProtocolError'>, <ProtocolError for 127.0.0.1/RPC2: 401 Unauthorized>: file: C:\Users\monitor\Desktop\app\src\env\lib\site-packages\supervisor\xmlrpc.py line: 560
2020-08-19 16:47:22,187 [INFO] supervisor config path: C:\Users\monitor\Desktop\app\src\service\supervisord.conf
2020-08-19 16:47:22,389 [INFO] supervisor starting...
2020-08-19 16:47:35,759 [INFO] supervisorctl shutdown
2020-08-19 16:47:37,272 [INFO] Server requires authentication
2020-08-19 16:47:39,292 [INFO] error: <class 'xmlrpc.client.ProtocolError'>, <ProtocolError for 127.0.0.1/RPC2: 401 Unauthorized>: file: C:\Users\monitor\Desktop\app\src\env\lib\site-packages\supervisor\xmlrpc.py line: 560
2020-08-19 16:47:58,498 [INFO] supervisor config path: C:\Users\monitor\Desktop\app\src\service\supervisord.conf
2020-08-19 16:47:58,701 [INFO] supervisor starting...

both service.log and service.conf are in same folder

Thanks for a quick response 👍

alexsilva commented 4 years ago

Also attach the supervisor logs (supervisord.log) and, if possible, this executable in a zip. Only trying to set up an environment like yours will I be able to find a solution.

tariksetia commented 4 years ago

Hi! Alex, At this point I don't have permission to share the executable. Is there any other way? attached are the logs you asked for. They are created in the directory specified in conf file. supervisord.log

2020-08-20 04:46:47,060 INFO process group added: 'pbswarn'
2020-08-20 04:46:47,091 INFO supervisord started with pid 4300
2020-08-20 04:46:47,718 INFO Spawned: 'pbswarn' with pid 6308
2020-08-20 04:46:48,748 INFO success: pbswarn entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

2020-08-20 04:50:55,475 INFO process group added: 'pbswarn'
2020-08-20 04:50:55,490 INFO supervisord started with pid 5176
2020-08-20 04:50:55,990 INFO Spawned: 'pbswarn' with pid 6260
2020-08-20 04:50:57,007 INFO success: pbswarn entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2020-08-20 04:52:16,335 INFO process group added: 'pbswarn'
2020-08-20 04:52:16,350 INFO supervisord started with pid 5084
2020-08-20 04:52:16,867 INFO Spawned: 'pbswarn' with pid 2468
2020-08-20 04:52:17,897 INFO success: pbswarn entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
// When stopped using web interface
2020-08-20 04:53:54,799 DEBG killing pbswarn (pid 2468) with signal SIGTERM
2020-08-20 04:53:55,315 INFO waiting for pbswarn to stop
2020-08-20 04:53:55,315 INFO stopped: pbswarn (exit status 1)
2020-08-20 04:53:55,315 DEBG fd <StreamAsync(stdout-pid-2468, started daemon 6072)> closed, stopped monitoring <PStreamOutputDispatcher at 2315263458368 for <Subprocess at 2315262901744 with name pbswarn in state STOPPED> (stdout)>
2020-08-20 04:53:55,315 DEBG fd <StreamAsync(stderr-pid-2468, started daemon 1216)> closed, stopped monitoring <PStreamOutputDispatcher at 2315263458848 for <Subprocess at 2315262901744 with name pbswarn in state STOPPED> (stderr)>
2020-08-20 04:53:55,315 DEBG fd <_io.TextIOWrapper name=5 encoding='cp1252'> closed, stopped monitoring <PInputDispatcher at 2315263598704 for <Subprocess at 2315262901744 with name pbswarn in state STOPPED> (stdin)>

When stopped using web interface, it only killed one of the four processes started by the executable. When stopped via services console and stop command, nothing popped up in the logs and all the four processes started by executable were still running

alexsilva commented 4 years ago

At least in the problem with the service, the supervisor control needs the authentication data to be able to execute the command because you have configured a username and password in the api.

[supervisorctl]

[inet_http_server]
port = 127.0.0.1:9001
username = admin
password = admin

Should be (Login is automatic in this case):

[inet_http_server]
port = 127.0.0.1:9001
username = admin
password = admin

[supervisorctl]
serverurl = http://127.0.0.1:9001
username = admin
password = admin

The service log reported this. Server requires authentication

2020-08-19 16:45:22,935 [INFO] Server requires authentication
2020-08-19 16:45:24,952 [INFO] error: <class 'xmlrpc.client.ProtocolError'>, <ProtocolError for 127.0.0.1/RPC2: 401 Unauthorized>: file: C:\Users\monitor\Desktop\app\src\env\lib\site-packages\supervisor\xmlrpc.py line: 560
tariksetia commented 4 years ago

I made this change, updated the service. Now service.log is showing different error relating to xmlrpc

2020-08-20 09:16:53,750 [INFO] supervisor config path: C:\Users\IEUser\Desktop\warn-processor-2.0\src\service\supervisord.conf
2020-08-20 09:16:53,968 [INFO] supervisor starting...
2020-08-20 09:18:43,549 [INFO] supervisorctl shutdown
2020-08-20 09:18:44,579 [INFO] error: <class 'xmlrpc.client.Fault'>, <Fault 1: 'UNKNOWN_METHOD'>: file: C:\Users\IEUser\AppData\Local\Programs\Python\Python38\lib\xmlrpc\client.py line: 655
tariksetia commented 4 years ago

May I use the email specified in your GitHub profile to share the executable with you?

alexsilva commented 4 years ago

It can help in that case. Put the configuration file and logs so I can analyze it better. Can you tell if the user who runs the service is an administrator (windows services can run with different users)?

Em qui., 20 de ago. de 2020 às 19:45, tariksetia notifications@github.com escreveu:

May I use the email specified in your GitHub profile to share the executable with you?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alexsilva/supervisor/issues/24#issuecomment-677943600, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPOFFR46H6ZJQNHLDHTRQDSBWRRHANCNFSM4QFOEYEQ .

tariksetia commented 4 years ago

shared the zip package via GDrive.

I am not sure about the who would be the window service user, But It will always be installed from powershell/CMD running in Administrator Mode.

alexsilva commented 4 years ago

Even I had difficulty because the failures were totally meaningless. Having understood the problems, I was able to correct them and make the use more intuitive, displaying fault messages that make more sense.

After updating the github supervisor, correct the executable paths in the configuration below that now there will be no more problems.

In case of doubt, use the official documentation. http://supervisord.org/configuration.html

[supervisord]
logfile=E:/superissue/svc/supervisord.log
logfile_maxbytes=50MB
logfile_backups=10
loglevel=debug
pidfile=E:/superissue/svc/supervisord.pid
nodaemon=false
stopwaitsecs=1
;identifier=superissue

[inet_http_server]
port=*:9003
username=pbs
password=warnp

[program:pbswarn]
command=E:/superissue/warn_processor.exe
directory = E:/superissue
# Exclusive process options
# http://supervisord.org/configuration.html?highlight=supervisorctl#program-x-section-settings
# Stops a process tree with the utility [taskkill /PID /F /T]
killasgroup = true
stopasgroup = true

# Supervisor communication interface (necessary to send command through the control).
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=http://127.0.0.1:9003
username=pbs
password=warnp
tariksetia commented 4 years ago

Thank You!!! Alex.. This is working.

alexsilva commented 4 years ago

Good. You help a lot too. Thanks.

tariksetia commented 3 years ago

@alexsilva Is the config above valid for using RPC? I am getting following error

server = ServerProxy('http://127.0.0.1:9999/RPC2') server.supervisor.getState() Traceback (most recent call last): File "", line 1, in File "C:\Users\tarik.setia.conda\envs\processor\lib\xmlrpc\client.py", line 1112, in call return self.send(self.name, args) File "C:\Users\tarik.setia.conda\envs\processor\lib\xmlrpc\client.py", line 1452, in request verbose=self.verbose File "C:\Users\tarik.setia.conda\envs\processor\lib\xmlrpc\client.py", line 1154, in request return self.single_request(host, handler, request_body, verbose) File "C:\Users\tarik.setia.conda\envs\processor\lib\xmlrpc\client.py", line 1187, in single_request dict(resp.getheaders()) xmlrpc.client.ProtocolError: <ProtocolError for 127.0.0.1:9999/RPC2: 401 Unauthorized>

alexsilva commented 3 years ago

You are trying to access the rpc server and for that you need to authenticate. You have to pass the username and password.

https://docs.python.org/3/library/xmlrpc.client.html

Both the HTTP and HTTPS transports support the URL syntax extension for HTTP Basic Authentication: http://user:pass@host:port/path. The user:pass portion will be base64-encoded as an HTTP ‘Authorization’ header, and sent to the remote server as part of the connection process when invoking an XML-RPC method. You only need to use this if the remote server requires a Basic Authentication user and password. If an HTTPS URL is provided, context may be ssl.SSLContext and configures the SSL settings of the underlying HTTPS connection.