search

adrpar / paqu

A parallel query engine for MySQL + Spider Engine built on a fork of shard-query
GNU General Public License v2.0
10 stars 0 forks source link

Mass jobs error #26

Open jochenklar opened 10 years ago

jochenklar commented 10 years ago

When creating a lot of jobs in a setup together with the mysql query queue, the following error appears ~ every 3 jobs.

Got error 10000 'Error on remote system: 2006: MySQL server has gone away' from FEDERATED

When you include a .5 s wait between the submissions, the errors do not appear.

Any thoughts?

kristinriebe commented 10 years ago

I also get occasional timeouts (Proxy Error, 502) even with wait-times in between:

Error while connection to server: Got response: 502 Proxy Error

Also, when many jobs are submitted, job handling becomes very slow (on the interface as well as via UWS). When this happens, I can see on the spider head node several processes like this:

SELECT qqueue_addJob(NULL, 260, 'user', 'short', '/* PaQu: QID 10661255 */ ....

that take quite long (at least 10 minutes, currently). So there may be a problem with the mysql_query_queue plugin.