search

flux-framework / flux-core

core services for the Flux resource management framework
GNU Lesser General Public License v3.0
163 stars 49 forks source link

job-list: default 'since' to something != 0 #5284

Open chu11 opened 1 year ago

chu11 commented 1 year ago

Something I was just thinking off randomly.

We default since to 0 to say "consider all jobs that are inactive historically" when filtering jobs. This isn't a huge deal when all inactive jobs are stored in memory like they are right now.

We'll move to a database of purged jobs someday in the future (#4336). Which could include like bazillions of jobs. We perhaps do not want to filter / constraint match on all of them.

As a somewhat pre-emptive / "prepare users" move to remove this worst case, perhaps we should default since to != 0 ... not sure what a good default would be. like 48 hours or 1 week ago? A warning could be output if we stopped searching b/c we hit the limit.

grondo commented 1 year ago

Another alternative would be to limit the number of matches (which I think we already do? max_entries=1000 or something...)

chu11 commented 1 year ago

Another alternative would be to limit the number of matches (which I think we already do? max_entries=1000 or something...)

Yeah, we already do that. The issue is if you have some obscure search and don't match anything (or very little). So you'd be scanning the entire database for potential matches, but not actually matching anything.

grondo commented 1 year ago

Ah, so your worry is users inadvertently creating searches that tax the database too heavily? I can see the sense in that, but it does seem like modern databases should be able to handle this kind of query better than we think.

My worry would be incorrect answers by default. For example, a user queries "give me all my jobs that ran on fluke123" and none have run in the past week on that host, so the answer is an empty list.

chu11 commented 1 year ago

My worry would be incorrect answers by default. For example, a user queries "give me all my jobs that ran on fluke123" and none have run in the past week on that host, so the answer is an empty list.

Ahhh I didn't think of it that way. That is a good point.

Ah, so your worry is users inadvertently creating searches that tax the database too heavily? I can see the sense in that, but it does seem like modern databases should be able to handle this kind of query better than we think.

Yeah, we'll have to see how the job constraints stuff ends up working out. Now that we're splitting out the job-archive into flux-acounting (https://github.com/flux-framework/flux-accounting/pull/357) we have less "dependency" and can tweak the database per our own needs. Given the initial job constraints implementation (#5126) and the initial job db in #4336, I think we'd just iterate through every inactive job in the database ... which is bad of course. (side note, #4336 has to be re-worked given job constraints.)

chu11 commented 8 months ago

By cahnce, I noticed that in sacct there are "time windows" with their searches. Only jobs that started in the specified time range (-S, -E) will be returned. That's a alternate way to deal with the general "don't scan the whole database".