Launch queue processes from server

[crism]

When a user runs start_queue.sh under their own userid, but the Web server runs under a different userid, processing fails. The PHP processes from the server side create job-specific directories owned by the Web server process, but the queue workers are running as the user. Thus, permission is denied when trying to write the output from a job.

Solution: kick off the queue workers from the server. We don’t have anything like a dashboard right now, but this could be the start of one.

[axfelix]

I like the sound of this. Out of curiosity, was this an issue you hit with your local configuration?

[crism]

@axfelix: Yes, it was. I am running the system on my laptop; Apache runs as www-data, but I’m logged in as (and ran start_queues.sh) as crism. The system (not sure which part) sets permissions on the queue directories to 755, which means that the Web server and the queue workers can’t talk with each other.

[crism]

A trivial, completely insecure proof-of-concept worked to start the queues, running as www-data. However, now when I upload a new document, it sits in limbo forever. It seems the new workers don’t see the queue somehow… why not? That user should be able to communicate with the database—after all, that’s how all the other database communication happens. Are the jobs paused somehow? Unfortunately, because they’re running within the Zend framework, there are no indications of error or activity unless a logging event is triggered…

[crism]

There were queue cached objects owned by crism. Clearing out var completely allowed the www-data-owned workers to work. Now, unoconv can’t fire up a listener… but that’s a new problem! Progress!

[crism]

Putting this aside for now. Manually starting queues as www-data is good enough for now, though we will likely want a control panel at some point.

[crism]

I did, however, fix another problem with unoconv running as a homeless user, in pursuit of this, so that’s now checked in. Hrm, except now it fails tests… why?

[axfelix]

does it pass in practice and just fail the tests?

[crism]

In practice, I get conversion results this way; however, I am very nervous about a change that doesn’t pass unit tests, especially if I don’t understand why. It means I can’t get a green dashboard for future changes either, so something may well inadvertently break. The not understanding it part is especially concerning…

[axfelix]

I was able to run it as a user without a home directory locally though it took two tries -- the first time it failed to start the listener (or so it thought).

Libreoffice 4.2.8 and unoconv 0.6.

[crism]

Ah, OK. I’ve figured out a few things. First (thanks @axfelix), the rc.d scripts provided with the installation are sufficient for running as www-data; my server-side hack is unnecessary.

Secondly, https://code.google.com/p/jodconverter/issues/detail?id=84 seems to have some answers. If one runs unoconv for the first time as a user, soffice.bin configures itself, then exits with status 81. So all these dodges I’ve been doing, trying to start in a clean environment, have been counterproductive. On our server, it’s running as www-data, and has, so it works. I think I now have that set up locally.

I’m closing this now.