service created by service.py never starts

[cdrandin]

• Django 1.7
• Python 2.7
• django-windows-tools 0.1.3
• Windows 2008

I have followed the guide for setting up celery services here.

I have adjusted parameters to work for my settings and still fails to start up properly when I check the task manager.

It shows I have a django-myapp-service created.

Here is my services.ini file:

[services]
# Services to be run on all machines
run=celeryworker
clean=C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celery.log

[BEATSERVER]
# There should be only one machine with the celerybeat service
run=celeryworker celerybeat
clean=C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celerybeat.pid;C:\inetpub\wwwroot\Django\BPTS\myapp\logs\beat.log;C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celery.log

[celeryworker]
command=celery worker
parameters=--app=myapp.celery -f C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celery.log -l info

[celerybeat]
command=celery beat
parameters=--app=myapp.celery -f C:\inetpub\wwwroot\Django\BPTS\myapp\logs\beat.log -l info --pidfile=C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celerybeat.pid

[runserver]
# Runs the debug server and listen on port 8000
# This one is just an example to show that any manage command can be used
command=runserver
parameters=--noreload --insecure 0.0.0.0:8000

[log]
filename=C:\inetpub\wwwroot\Django\BPTS\myapp\logs\service.log
level=INFO

Taking apart these commands and parameters I manually input them

python manage.py celery worker --app=myapp.celery -f C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celery.log -l info

python manage.py celery beat --app=trackingsystem.celery -f C:\inetpub\wwwroot\Django\BPTS\trackingsystem\logs\beat.log -l info --pidfile=C:\inetpub\wwwroot\Django\BPTS\trackingsystem\logs\celerybeat.pid

and they work properly.

[mrbean-bremen]

Is this also something you had running before? Note that I haven't used celery for a long time (as I wrote in #19), so I don't really have the experience, but it looks as if something with the service is going wrong. Do you see something in the system event log? Are you sure that the python path (and other needed paths) are in the system path?

[cdrandin]

I have included --pythonpath=C:\inetpub\wwwroot\Django\BPTS\myapp for the parameters in the service.ini file, but still fails to start.

I took a look at the Event Viewer and found nothing on the django service django-myapp-service.

Also, the service.ini can handle the runserver command so then I can guess it is aware of python manage.py at that directory. I don't see why I would have to give it a pythonpath again.

How does service.py know how to tell Windows services to use a specific manage.py at my current directory.

Expanding the command to be abs path. The follow does work.

C:\Python27\python.exe C:\inetpub\wwwroot\Django\BPTS\myapp\manage.py celery worker --pythonpath=C:\inetpub\wwwroot\Django\BPTS\myapp

[mrbean-bremen]

Sorry, I didn't mean the pythonpath, but the path to the python executable and scripts (in the system PATH, which is used by the service). I don't think that this is the problem, but it is always a good idea to check the obvious first.

[cdrandin]

Yes C:\Python27;C:\Python27\Scripts\; is in the system path.

I feel everything should be referenced properly. I already have this Django app live on a production server, but I am not sure why the service won't start up. Windows won't tell me why it fails to start up since it is a 3rd party service and doesn't keep recorded (at least what it says when I try to start the service).

[mrbean-bremen]

If you had it already running, that is strange. Do you use the same Celery version as before? Any other package version changed?

[cdrandin]

Nothing else has changed. My only issue is that the service simply refuses to run celery. For service.ini how does it know it should run python manage.py at C:\inetpub\wwwroot\Django\BPTS\myapp\ when using python service [options]? I was looking into the Service Manager and was trying to see how it would serve the service I gave it. There is no info on reference of what service.ini it would be running.

And like I've said before everything is set up properly celery and the broker, RabbitMQ which I am using, and have been testing it by manually running the commands myself.

If the service with windows just isn't working I figured the next thing I could do is to use subprocess.Pcall to run celery worker and beat, but it seems so hack, but it really is the only thing I can think of to just get it working.

By any chance were you able to get an example service up and running with the runserver service?

Like:

[runserver]
# Runs the debug server and listen on port 8000
# This one is just an example to show that any manage command can be used
command=runserver
parameters=--noreload --insecure 0.0.0.0:8000

[mrbean-bremen]

I had that running several years ago with Django 1.6, python 2.7 and django-windows-tools 0.1.2, using django-celery. Since than I haven't tried it, and had no time no check it this week. I may try something this weekend, but can't promise it.

[cdrandin]

I was able to find some logs about the django-myapp-service

Here is an error I was able to find.

The instance's SvcRun() method failed 
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\win32\lib\win32serviceutil.py", line 835, in SvcRun
    self.SvcDoRun()
  File "C:\Python27\lib\site-packages\django_windows_tools\service.py", line 245, in SvcDoRun
    self.start()
  File "C:\Python27\lib\site-packages\django_windows_tools\service.py", line 278, in start
    self.processes = start_commands(self.config)
  File "C:\Python27\lib\site-packages\django_windows_tools\service.py", line 152, in start_commands
    services = config.get(node_name, 'run') if config.has_section(node_name) else config.get('services', 'run')
  File "C:\Python27\lib\ConfigParser.py", line 618, in get
    raise NoOptionError(option, section)
NoOptionError: No option 'run' in section: 'services' 
%2: %3```

[mrbean-bremen]

Are you using the same config file as you posted above? The error message states that you miss the 'run' line under 'services'. I just did a quick check with runserver - I didn't get it to run yet, probably due to a probem with the virtual enviroment, but the config file did get parsed without problems. The error messages show up in the event log under Applications, not System (you probably already found that). Update: if I fix the virtualenv problem (a bug in multiprocessing) the runserver process runs as service. As you don't use a virtual environment, this should not a problem for you. I tested this with Python 2.7.13, Django 1.7 and django-windows-tools 0.1.3.

[cdrandin]

I am using the same service.ini file above when running python service.py in the same directory as C:/.../myapp/.

I am not using a virtualenv. You are using all the same versions I am for. I am not sure why I am getting this problem.

[mrbean-bremen]

Can you try to run it outside of the service, using: pythonservice.exe -debug myapp Does this work?

[cdrandin]

pythonservice.exe doesn't exist for me.

[mrbean-bremen]

Ah, sorry, it is in the win32 folder (lib/site-packages/win32 or similar). This is used to run the service. You have pywin32 installed, right?

[cdrandin]

Added C:\Python27\Lib\site-packages\win32;C:\Python27\Lib\site-packages\pywin32_system32; and it worked.

I got the same error:

Debugging service django-trackingsystem-service - press Ctrl+C to stop.
Info 0x400000FF - Initialization
Info 0x400000FF - C:\inetpub\wwwroot\Django\BPTS\trackingsystem\service.ini
Info 0x400000FF - starting
Error 0xC0000003 - The instance's SvcRun() method failed

Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\win32\lib\win32serviceutil.py", line 835,
in SvcRun
    self.SvcDoRun()
  File "C:\Python27\lib\site-packages\django_windows_tools\service.py", line 245
, in SvcDoRun
    self.start()
  File "C:\Python27\lib\site-packages\django_windows_tools\service.py", line 278
, in start
    self.processes = start_commands(self.config)
  File "C:\Python27\lib\site-packages\django_windows_tools\service.py", line 152
, in start_commands
    services = config.get(node_name, 'run') if config.has_section(node_name) els
e config.get('services', 'run')
  File "C:\Python27\Lib\ConfigParser.py", line 618, in get
    raise NoOptionError(option, section)
NoOptionError: No option 'run' in section: 'services'

(null): (null)

Future reference (for anyone):

pythonservice.exe -debug django-myapp-service

[cdrandin]

This is my part of myservice.ini file, mainly where it complains.

[services]
# Services to be run on all machines
run=celeryworker
clean=C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celery.log

[BEATSERVER]
# There should be only one machine with the celerybeat service
run=celeryworker celerybeat
clean=C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celerybeat.pid;C:\inetpub\wwwroot\Django\BPTS\myapp\logs\beat.log;C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celery.log

[celeryworker]
command=celery worker
parameters=--pythonpath=C:\inetpub\wwwroot\Django\BPTS\myapp --app=myapp.celery -f C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celery.log -l info

[mrbean-bremen]

Future reference (for anyone) pythonservice.exe -debug django-myapp-service

Ah yes, sorry - this needs the service name.

This is my server.ini file.

You mean service.ini, I guess. Is this the whole file? Do you have a log section (not sure if it is needed, but just in case...)?

[cdrandin]

Yes, service.ini. Sorry.

[cdrandin]

This is the entire service.ini:

[services]
# Services to be run on all machines
run=celeryworker
clean=C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celery.log

[BEATSERVER]
# There should be only one machine with the celerybeat service
run=celeryworker celerybeat
clean=C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celerybeat.pid;C:\inetpub\wwwroot\Django\BPTS\myapp\logs\beat.log;C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celery.log

[celeryworker]
command=celery worker
parameters=--pythonpath=C:\inetpub\wwwroot\Django\BPTS\myapp --app=myapp.celery -f C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celery.log -l info

[celerybeat]
command=celery beat
parameters=--pythonpath=C:\inetpub\wwwroot\Django\BPTS\myapp --app=myapp.celery -f C:\inetpub\wwwroot\Django\BPTS\myapp\logs\beat.log -l info --pidfile=C:\inetpub\wwwroot\Django\BPTS\myapp\logs\celerybeat.pid

[runserver]
# Runs the debug server and listen on port 8000
# This one is just an example to show that any manage command can be used
command=runserver
parameters=--pythonpath=C:\inetpub\wwwroot\Django\BPTS\myapp --noreload --insecure 0.0.0.0:8000

[log]
filename=C:\inetpub\wwwroot\Django\BPTS\myapp\logs\service.log
level=INFO

No logs ever make it to C:\inetpub\wwwroot\Django\BPTS\myapp\logs\

[mrbean-bremen]

Have you tried to replace run=celeryworker with run=runserver for testing?

[mrbean-bremen]

Also - I'm not sure if command can consist of 2 commands (e.g. celery worker), is that correct? Shouldn't worker rather be an argument?

[cdrandin]

Also - I'm not sure if command can consist of 2 commands (e.g. celery worker), is that correct?

I will check. I run celeryd and was a depreciation thing, so figured I'd change it.

[mrbean-bremen]

Anyway, I still don't understand the error. It states that it can't find the run option in services that definitely exists. Either it reads the wrong file for some reason (no idea why), or the file is somehow damaged (extra invisible characters, incorrect encoding or similar).

[mrbean-bremen]

You can attach your zipped ini file if you want so I can check if it works for me (at least for runserver).

[cdrandin]

Yeah, I tried using runserver service in places of the service run. Still resulting in parsing error.

service.ini

[mrbean-bremen]

hm, the download seems to be stuck... can you just zip the file and drag it into the comment?

[cdrandin]

Ah sorry.

service.zip

[mrbean-bremen]

Ok, now we are getting somewhere. I get the same error, but if I convert the line ending in the ini file to Windows (or to Unix, for that matter), it works. There seem to Mac line endings for some reason which the parser cannot work with.

[cdrandin]

Haha oh geez. Well, that pretty much sums up that fiasco. Everything works properly now. Thanks a bunch for your help through the debugging process.

[mrbean-bremen]

Glad that I could help :)

[cdrandin]

Real quick. When playing with celery more. I don't think the beat service is actually running. Should I be running a separate service.ini that solely handles starting up the beatserver so only 1 is ever active?

I am just wondering, what do I do with the service [BEATSERVER]. It is there, but nothing is done about it. Where would be a good place to separate the global beatserver from the rest of the Django apps and other apps that would use it?

Edit:

Nvm got it.

python winservice_install --beat (though unsure what --beat-machine actually does. I gave it a name, but I can't seem to find a reference to it).

then do the rest of the steps to install, start.

Though I will say, doing it as such python winservice_install --beat wouldn't the celery beat dependent on this Django app instead of just being completely separate?

Still curious if there is a better way to make it independent from my apps.

[mrbean-bremen]

As far as I remember, I used it the same way. As I wrote, I don't use it for several years now, so there is not much I can help you with here.