search

galaxyproject / pulsar

Distributed job execution application built for Galaxy
https://pulsar.readthedocs.io
Apache License 2.0
37 stars 50 forks source link

Pulsar installation on Windows... #121

Open edwardsnj opened 8 years ago

edwardsnj commented 8 years ago

I'm having trouble getting to square one with a Windows deployment of pulsar. I'm working towards a cloud-based (AWS cloud formation, auto-scaling, ideally) windows instance that can act as a worker for a CloudMan galaxy cluster to carry out mass-spec conversion utilities which must run on windows.

I'm following the published installation docs, using virtualenv and pip. I'm using EPD Python corresponding to 2.7.11 on a clean, new (virtual) 64-bit Windows 7 installation.

Install with pip in the virtual environment seems to go fine. Pulsar-config with no arguments runs without any errors, but app.yaml is essentially empty, with just "---" in it. server.ini is created too - this looks correct, although I'm worried that there are so many things in there I don't plan to use. Should I comment out most of it?

There is no runable pulsar.exe, but I do see pulsar-main.exe and pulsar-run.exe. Pulsar-main.exe doesn't seem to accept the --url argument (referenced later in the docs) so I guess pulsar-run.exe is the Windows executable for pulsar.

The galaxy.utils import is problematic, since my Windows/EPD python has no module "grp" (for Linux groups). I wrapped this in a try/except ImportError block with a pass on exception to proceed.

Both pulsar-run and pulsar-check generate the following error when I run them:

File "c:\users\nathan\pulsar\venv\lib\site-packages\pulsar\client\transport\standard.py", line 49, in execute raise PulsarClientTransportError(transport_message=exc.reason) pulsar.client.exceptions.PulsarClientTransportError: Unknown transport error (transport message: [Errno 10061] No connection could be made because the target machine actively refused it)

Thanks,

natefoo commented 8 years ago

This was only partially fixed by galaxyproject/galaxy#2928 (the import grp issue)

JenCabral commented 8 years ago

We also don't see the pulsar.exe file either. We copied over server.ini and app.yml from pulsar\ to pulsar\venv\Lib\site-packages and were able to get puslar-main.exe to run but we still get the same error as original poster when we run pulsar-check.

Also, when we attempted to run the pulsar-main.exe with -d option we got the following error: Traceback (most recent call last): File "c:\python27\Lib\runpy.py", line 174, in _run_module_as_main "main", fname, loader, pkg_name) File "c:\python27\Lib\runpy.py", line 72, in _run_code exec code in run_globals File "C:\Program Files\pulsar\venv\Scripts\pulsar-main.exemain.py", line 9, in File "c:\progra~1\pulsar\venv\lib\site-packages\pulsar\main.py", line 279, in main raise ImportError(REQUIRES_DAEMONIZE_MESSAGE) ImportError: Attempted to use Pulsar in daemon mode, but daemonize is unavailable.

JenCabral commented 8 years ago

Tried this again with 0.7.3 and am still receiving the same error as above. Is there a way to get this working on windows server?

edwardsnj commented 8 years ago

I have had more success with downloading the latest tagged source release (0.7.3) zipfile and following the install instructions here. I use the virtualenv version of the install, give the explicit path to my python installation's venv command, and execute pulsar using the run.bat script. I was even able to configure https to work (I created the certificate under Linux and it worked without any additional installation, I think, on Windows). So far, so good.

To test, start pulsar with run.bat, then at another command-line, execute the run_client_tests.py using the .venv\bin\python.exe binary.

I guess the open aspect of this ticket is just making sure that all of the installation instructions work for Windows as well as Linux.

Hope this helps...

natefoo commented 7 years ago

@jmchilton I'm out of my depth on this one. Any ideas?

jvolkening commented 6 years ago

This one's been open for a while, but I'm running into the same issues (No connection could be made because the target machine actively refused it) trying to get Pulsar running on an EC2 Windows Server 2016 instance. I followed the docs using pip and made it through pulsar-config. Are there any updated guides or advice for running Pulsar on Windows?