msavin / SteveJobs

A simple jobs queue that just works (for Meteor.js)
Other
207 stars 35 forks source link

publish collection for GUI updates #60

Closed tosinek closed 5 years ago

tosinek commented 6 years ago

Hi, I may have a use case for direct DB access.

When a user logs in, the code starts periodic data refresh and the data is synced with Mongo. Each user queries their own data from SQL. I would like to display in GUI e.g. when the next update is due. There is quite complex logic on what data and when to sync (there is several layers of updates) and I don't have a good solution for it (I guess I would have to start querying the server periodically using methods). Instead, I would like to subscribe to all user's scheduled jobs and that way, I would always know what has been scheduled.

(my current "solution" is that the client contains code with Meteor.setInterval that triggers some of the jobs without scheduling them upfront. This gets too messy, as there are situation, when the server can actually schedule something on users' behalf (data driven) but there is no easy way how to tell the client)

The "jobs_data" collection seems to be scoped to a file, and Meteor won't let me create another collection/subscription (on the client). Could you expose the collection variable in a way I would be able to create a publication for it? Or is there a workaround that would not require your help? (the Jobs.collection is available only on server and I can't create minimongo collection on the client)

Also, with the UI widget, where is its code? I could only find a minified file in my Meteor packages (which is hard to read). How does it handle user permissions? Or can everyone open the job schedule and start messing up with jobs? I would like to be able to edit the table and add e.g. "who triggered this job" column, as my arguments are currently an object, not individual strings.

msavin commented 6 years ago

I think this might answer your question:

1- Jobs Debugging UI for development purposes only. I will have a production option soon. 2- Jobs.collection() gives you direct access to the Jobs collection, which you can use in any Meteor publication

On Fri, Jul 27, 2018 at 2:17 PM, tosinek notifications@github.com wrote:

Hi, I may have a use case for direct DB access.

When a user logs in, the code starts periodic data refresh and the data is synced with Mongo. Each user queries their own data from SQL. I would like to display in GUI e.g. when the next update is due. There is quite complex logic on what data and when to sync (there is several layers of updates) and I don't have a good solution for it (I guess I would have to start querying the server periodically using methods). Instead, I would like to subscribe to all user's scheduled jobs and that way, I would always know what has been scheduled.

(my current "solution" is that the client contains code with Meteor.setInterval that triggers some of the jobs without scheduling them upfront. This gets too messy, as there are situation, when the server can actually schedule something on users' behalf (data driven) but there is no easy way how to tell the client)

The "jobs_data" collection seems to be scoped to a file, and Meteor won't let me create another collection/subscription (on the client). Could you expose the collection variable in a way I would be able to create a publication for it? Or is there a workaround that would not require your help? (the Jobs.collection is available only on server and I can't create minimongo collection on the client)

Also, with the UI widget, where is its code? I could only find a minified file in my Meteor packages (which is hard to read). How does it handle user permissions? Or can everyone open the job schedule and start messing up with jobs? I would like to be able to edit the table and add e.g. "who triggered this job" column, as my arguments are currently an object, not individual strings.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/msavin/SteveJobs..meteor.jobs.scheduler.queue.background.tasks/issues/60, or mute the thread https://github.com/notifications/unsubscribe-auth/AB7-g7zNco6Q27Kvau1G7HawiscbL729ks5uKwT2gaJpZM4VjZ0L .

tosinek commented 6 years ago

Thank you for your reply Max. I seem to be unable to subscribe to the publication, so while I can create it, there is no minimongo counterpart that would add the subscription data?

PS: I am creating another thread for an issue with UI package.

ZhaopingLuo commented 6 years ago

Hi Max, I have the same issue. Yesterday after I deployed, I found that the data is not in my local minimongo anymore. Because of that, my jobs GUI was broken :(

Then I realized that it is because of my deployment doesn't allow this debugger: msavin:sjobs-ui-blaze, but the minimongo is generated from the debugger. I tried to search the sjob-ui-blaze's code to learn how to create the minimongo but didn't find it.

Could you give me any suggestion?

ZhaopingLuo commented 6 years ago

Ah, never mind I've got it. I publish by: Jobs.collection subscribe by: export const JobsCollection = new Mongo.Collection('jobs_data'); But need to remove the debugger first, otherwise, there will be a conflict.

charlesdeb commented 5 years ago

I am still a little puzzled by this. Just how can I get access to the job_data collection while on the client. I want to wire up a couple of jobs start/jobs stop buttons in the admin area of my UI, and can't access Jobs.collection on the client. It looks like @ZhaopingLuo has solved this, but I don't quite follow his subscription example. I could do it with meteor methods, but that's a bit clunky - and I haven't quite worked out yet how to make method calls reactive.

charlesdeb commented 5 years ago

In the end, I figured out using reactive methods and have wrapped some of the Jobs functionality that only appears to run on the server in restricted methods that I am calling from the client.

msavin commented 5 years ago

@charlesdeb yes, you can also do publications using Jobs.collection - this isn't done by default because it could be a security risk

charlesdeb commented 5 years ago

I would much prefer to use a Job.collection subscription rather than the clunky methods I have cooked up, but couldn't figure out how to get what @ZhaopingLuo was saying to actually work. I think I did what he suggested (although I don't think I got rid of your UI - mostly because I actually quite like it!) but I could never get access to the Jobs collection on the client. A bit of sample code would be very handy if you could supply it!

joephuz commented 5 years ago

Im still having some weird issues here... (forgive me if this should be a new issue or even if should be asked on the mongol page) I tried the solution @ZhaopingLuo provides, which did in fact work to read the jobs on the client, but only when I (reluctantly) removed the debug UI. But now mongol is completely messed up too - now only showing the jobs_data, Account, users, meteor_Facts_server collection, all other collections have disappeared from mongol. I tried re-installing mongol with no luck.

Any ideas how to get mongol back?

it would still be a nice to have a better way to access the pending jobs in the client and still use the cool debug UI... I basically want to show a 'pending' graphic for videos as their thumbnails process using steve jobs..

joephuz commented 5 years ago

Update: After removing all the code related to: export const JobsCollection = new Mongo.Collection('jobs_data'); and a number of restarts, mongol did inexplicably repopulate the collections... But, I then added the above code back in, and now jobs_data is not appearing in mongol. Certainly not the end of the world, just thought I would update.

Thanks for all your hard work this is a great package! and Mongol too!