Open luixal opened 7 years ago
What environment is your worker code running in? Meteor (server) or node.js? Under meteor, jobcollection doesn't do anything special beyond what any other Meteor collection does regarding connection handling to the MongoDB. In fact it is using the same code. Under plain node.js, you are responsible for handling all MongoDB connection issues yourself, as you would with any other node.js app that connects to a database. Hope that helps.
I'm running two different mongo apps:
settings.json
like this:{
"public": {
"version": "0.0.1",
"environment": "production"
},
"env": {
"PORT": "3333",
"MONGO_URL": "mongodb://10.130.52.39:27017/app"
}
}
I guess the problem is that, eventually, the connection to the mongodb gets broken and causes that exception (which will be not problem, I would only prefer to catch the exception and generate a custom log), but I think (can't asure this, but 99%) that the jobServer (or the whole second meteor app) doesn't try to re-connect.
Any ideas?
Hi, okay there are two issues here that I can see.
1) You are using a remote MongoDB instance for the backing DB for your job-collection "app". As you note, this is not a problem in itself, but connection management issues will eventually arise (as you are seeing). All I can really say about it is that a jobCollection instance under these circumstances is identical to a regular old Meteor Collection. Anything you would need to do to keep that DB connection healthy for a Meteor Collection, you will also need to do for a jobCollection. There is no difference.
2) SInce you are using JobQueue to manage your workers (which is a level of abstraction higher than the jc.getWork()
mechanism is it built upon), you give up some flexibility and control. Your jobQueue instance calls jc.getWork()
internally (via a Meteor method call) and when handled on the server, it is that method call that interacts with MongoDB via the underlying Meteor Collection.
Since the JobQueue (returned by processJobs()
) is a long running async process meant to survive transient errors, etc, and meant to run remotely from the server it is connected to (e.g. in a vanilla node.js app, without Meteor), it does not assume or have access to any of the Meteor error handling features. Its only interaction with the server is via the aforementioned Meteor method generated by getWork()
, which can fail in a number of ways.
Right now, JobQueue simply logs such failures using console.error()
. The code for this is here: https://github.com/vsivsi/meteor-job/blob/b7b87d0d81f38f4eac5d67f0e8280277adc954a2/src/job_class.coffee#L154
This isn't ideal if you would like to capture these errors and log them elsewhere or otherwise act on them. I can think of a couple of ways to remedy this issue. One you can do without my help is to create a custom version of the JobQueue
class that does something else at the linked line of code above. That's not great, because you would essentially be forking that class, but it would work.
I'd be happy to add some functionality to JobQueue to try to cover this case. Here are some ideas:
jc.processJobs()
could accept an optional callback function that is invoked anytime there is an internal error, and if no callback is provided the current behavior will continue. This would be the simplest to implement, but isn't terribly flexible.
JobQueue (returned by jc.processJobs()
) could become an event emitter, allowing you to register a handler for various events, the first of which could be an 'error'
event covering this case. This is obviously much more flexible for handling other future types of events, etc. But is a bit more work to implement.
Thoughts on this?
@luixal I have prototyped the first idea above: adding an optional error callback to processJobs
that receives any runtime errors from processJobs
and silences the default console.error
output. It is implemented on this branch, but I haven't yet integrated it into the meteor package.
https://github.com/vsivsi/meteor-job/tree/errorCallback#q--jobprocessjobsroot-type-options-worker
These changes are now also on the errorCallback
branch of the main meteor-job-collection project. https://github.com/vsivsi/meteor-job-collection/tree/errorCallback
Will try including the errorCallback
branch providing a simple logs to see the info dropped when I get the error (not surely when it will happen) and come back here.
I guess the problem is that, when an error closes/hangs the connection, job server gets stuck and I would need to reestablish it and restart the job server, which I could perfectly do from this callback.
Thanks!
Okay, let me know how it goes. Once you confirm that this solution works for you I can publish the changes to npm/Atmosphere within a day.
Hi,
I'm running JobCollection using a remote MongoDB (they run on different machines inside a local network) and, from time to time, I'm getting this exception thrown:
Any idea on how to solve or handle this?
thanks!