seomoz / qless

Queue / Pipeline Management
MIT License
294 stars 76 forks source link

Problems running web UI on startup #276

Open NathanBaulch opened 6 years ago

NathanBaulch commented 6 years ago

I've been trying unsuccessfully to get qless-web to run automatically on startup in Ubuntu 16.04 using either systemd or supervisord. I've tried all kinds of config tweaks but they all ultimately end with the service disappearing from the systemctl list or switching to FATAL status in supervisorctl. It works just fine if I run it manually from CLI.

Here's my very simple systemd file:

[Unit]
Description=Qless Web UI
After=redis-server.service

[Service]
User=ubuntu
ExecStart=/usr/bin/ruby /usr/local/bin/qless-web

[Install]
WantedBy=multi-user.target

Doing a daemon-reload && restart yields the following uninteresting syslogs:

systemd[1]: Reloading.
systemd[1]: Started ACPI event daemon.
systemd[1]: Reloading.
systemd[1]: Started ACPI event daemon.
systemd[1]: Started Qless Web UI.
ruby[1588]: [2018-03-19 15:18:12 +1100] Starting 'qless-web'...
ruby[1588]: [2018-03-19 15:18:12 +1100] trying port 5678...
ruby[1588]: Couldn't get a file descriptor referring to the console

I see the same file descriptor message when running manually, so I doubt that's it.

I found another log file ~/.vegas/qless_web/qless_web.log which contains:

Running with Rack handler: Rack::Handler::Thin
Thin web server (v1.7.2 codename Bachmanity)
Maximum connections set to 1024
Listening on 0.0.0.0:5678, CTRL+C to stop
log writing failed. can't be called from trap context

Is something preventing the real underlying issue to be logged?

The only thing that might be different on this machine is it has Ruby 2.3.0 (via apt-get) installed. Could that be it?

How can I troubleshoot this further? Thanks!

dlecocq commented 6 years ago

That's a good question. Cursory googling suggests that it's from trying to log from a signal handler. I don't have any experience with systemd, unfortunately.

If this is a machine where you can muck around a bit, in https://github.com/resque/resque/issues/1493 the reporter of the issue stitched in backtrace reporting into the standard logging module, which would at least help us track down where in our code it's throwing.

I imagine if we can figure out where it's throwing, we could probably figure out what signal it's receiving and then that might help to debug the systemd config.

stephenreay commented 5 years ago

I don't know if this is still an issue, but I found that running in Foreground mode, and suppressing the "open a browser" option help immensely. My unit file is:

[Unit]
Description=Qless Web UI
After=redis-server.service

[Service]
User=qless-web
ExecStart=/usr/local/bin/qless-web -L -F --env production
Type=simple

[Install]
WantedBy=multi-user.target
NathanBaulch commented 5 years ago

Thank you @stephenreay, that -F did the trick!

stephenreay commented 5 years ago

So, now I have my own issue related to this while trying to deploy to a production environment - there is no apparent (and certainly none documented) way to pass a redis connection string/config to the qless-web script.

It does accept a filename as the first argument, and will try to load it - but there is no real clue for non-ruby developers how to use this to configure a different connection than the default.

The most annoying part of this: in the qless-web script, there is a commented (I guess this script originated in the Resque project?) section which (if applied to Qless-Web) would do exactly what I want:

# opts.on('-r redis-connection', "--redis redis-connection", "set the Redis connection string") {|redis_conf|
#   runner.logger.info "Using Redis connection '#{redis_conf}'"
#   Resque.redis = redis_conf
# }

Any suggestions? Does anyone use this in a non-toy environment without hosting it inside their own ruby app as documented in the README (https://github.com/seomoz/qless#web-interface) ??

For reference: I need to be able to set the hostname/IP, protocol (i.e. TLS, and thus CApath), and DB number.