search

grisu / gricli

Grisu commandline client
7 stars 2 forks source link

"status" command results in error with too many jobs #234

Open yhal003 opened 12 years ago

yhal003 commented 12 years ago

our old friend "413 Entity Too Large" happens because method runs for too long.

makkus commented 12 years ago

What do you suggest? Kinda hard to solve this....

yhal003 commented 12 years ago

can we break request in batches of (say) 50 jobs per call?

On Mon, Apr 2, 2012 at 6:00 PM, Markus Binsteiner < reply@reply.github.com

wrote:

What do you suggest? Kinda hard to solve this....

Reply to this email directly or view it on GitHub: https://github.com/grisu/gricli/issues/234#issuecomment-4872775

makkus commented 12 years ago

Not easily at all. I'd say this should be considered if we decide to (re-)add batch support to gricli/grisu, but on its own I think it'd be too big a change for only a limited number of users. Those users could for example use a local backend, which would speed up things for them anyway, since a local backend is quicker than a ws-based one...

vladimir-mencl-eresearch commented 12 years ago

I remember I was getting that one quite a few times when running the large jobs. Would vote to have a proper fix ... when Gricli was managing a large number of jobs, things were falling apart (random errors). But haven't tried running a large batch recently....

makkus commented 12 years ago

Like I said: we'd need to implement proper, stable batch support. At the moment we are using loads of single jobs created by outside scripts to deal with batches of jobs. It's just not possible to cater for that in a way that is viable.

If we had batch support in Grisu we could "hide" them in the list of jobs and only list the "parent" job. And get more details on that if necessary. But until we have that, having too many (whatever that number is) jobs active in grisu is just not supported and something like this is nowhere to be seen in the nesi milestones. Happy to implement batch support. But, as always, it's a matter of priorities.

vladimir-mencl-eresearch commented 12 years ago

Aha. OK, I agree proper batch support would be the real solution.

I'm just not sure we should settle for saying "Grisu doesn't support large amounts of jobs" - that just makes our infrastructure flaky....

yhal003 commented 12 years ago

We did a lot of work to make backend stable. It should support 10,000 jobs per day or more without that much trouble. and gricli is fine to work with thousands of jobs too even now, it is just the status command...

On Thu, Apr 19, 2012 at 4:31 PM, vladimir-mencl-eresearch < reply@reply.github.com

wrote:

Aha. OK, I agree proper batch support would be the real solution.

I'm just not sure we should settle for saying "Grisu doesn't support large amounts of jobs" - that just makes our infrastructure flaky....

Reply to this email directly or view it on GitHub: https://github.com/grisu/gricli/issues/234#issuecomment-5214731

makkus commented 12 years ago

Ah, right. I see. Sorry, misunderstood.

Totally forgot about this command :-)

Yes, I think that should be possible. Will do.

makkus commented 12 years ago

Hm. Actually. Thinking about it, not all that easy, will require some change to the serviceinterface.

What about having a status command in the API? I guess that would be useful, and it could be processed on the backend itself. Might have to play with how to implement it (wether to use cached job statuses and such), but that would be easier....

yhal003 commented 12 years ago

but that command would still take a lot of time when user has lots of jobs. Or you want to have some "status" state that gets updated when user calls other methods, and "status" method just returns that value/data structure?

On Thu, Apr 19, 2012 at 4:55 PM, Markus Binsteiner < reply@reply.github.com

wrote:

Hm. Actually. Thinking about it, not all that easy, will require some change to the serviceinterface.

What about having a status command in the API? I guess that would be useful, and it could be processed on the backend itself. Might have to play with how to implement it (wether to use cached job statuses and such), but that would be easier....

Reply to this email directly or view it on GitHub: https://github.com/grisu/gricli/issues/234#issuecomment-5214920

makkus commented 12 years ago

Not sure I understand what you mean. You are saying, whenever another call is made (or every 5 minutes), all job statuses should be updated, and when the status call is made, only a current snapshot of all jobs (with partly cached/outdated statuses) is used?

yhal003 commented 12 years ago

yeah, maybe not every 5 minutes, but on event notifications (we do have working event notifications now, right?)

On Fri, Apr 20, 2012 at 8:23 AM, Markus Binsteiner < reply@reply.github.com

wrote:

Not sure I understand what you mean. You are saying, whenever another call is made (or every 5 minutes), all job statuses should be updated, and when the status call is made, only a current snapshot of all jobs (with partly cached/outdated statuses) is used?

Reply to this email directly or view it on GitHub: https://github.com/grisu/gricli/issues/234#issuecomment-5231138

makkus commented 12 years ago

On 20/04/12 08:36, yhal003 wrote:

yeah, maybe not every 5 minutes, but on event notifications (we do have working event notifications now, right?)

Not sure, if, then only for GT5Submitter. Would require a change to the JobSubmitter interface I guess.