search

abrt / retrace-server

Application for remote coredump analysis
GNU General Public License v2.0
40 stars 30 forks source link

bugzilla-query: Use 'limit' option and search only recently modified … #450

Closed DaveWysochanskiRH closed 2 years ago

DaveWysochanskiRH commented 2 years ago

…bugs

Set the 'limit' option to bzapi.build_query() function to ensure we are not artificially limited to a small default number of bugs. Note the 'limit' option is new in python-bugzilla 3.1.0. In production we were seeing an artificial limit of 20 bugs get queried without setting this option. If we ever meet our defined limit of 1000 bugs in the query this means we probably did not query all bugs.

In addition, cut down on the number of bugs in the query by only querying those with recent modification times. Here we pick 7 days, somewhat arbitrarily, but with the understanding that the cronjob that runs bugzilla-query should be run at least once per week. We can look at only the recently modified bugs because we are looking for additions of retrace-server task strings in the bug comment text.

Signed-off-by: Dave Wysochanski dwysocha@redhat.com

DaveWysochanskiRH commented 2 years ago

Ok, in 53cb39e I've fixed up the query to use "offset" as well as "limit" in the bugzilla API query, which is the way to query until all bugs are returned. I moved things around a bit and hopefully it is all more readable. As far as I can tell this does exactly what we want, and it should be safe for any future changes on the server end to limit the number of bugs returned.

A couple comments:

  1. This removes the intermediate query to bugzilla which only was querying the bug_ids. It's not clear if this was necessary before due to maybe some API limitation, but it's not necessary anymore.
  2. Query with increasing offsets until we get 0 bugs back from the query. The "while True" may look a bit odd but what this type of loop calls for is a "do .. while" and as far as I know this does not exist in Python. The paranoia in me wondered about the possibility of an infinite loop here, and I almost put in a second "break" out of the loop with some "failsafe" code to count the iterations through the loop and break. However, I think probably that is overkill and I didn't think it would see positive review so I left it out
  3. I added BugzillaQueryLastChangeDelta which represents the limit on bz modified times for the bugzilla query, but left the "limit" number hardcoded at 500, as I couldn't see how the "limit" number adds value to be made a parameter. In fact, we could probably omit the limit but then the internal limit of 20 would come into play and there would be many queries.
  4. I reformatted a number of lines towards the end to fit in 79 columns and conform to the python PEP-0008
DaveWysochanskiRH commented 2 years ago

I decided to split out the reformatting changes into a separate commit on top of the main commit which implements the functional changes of this PR. I am not sure how important cleanups like the line lengths are so if they are not important we can just drop that without affecting the functional changes.

DaveWysochanskiRH commented 2 years ago

The two remaining items have been addressed in c09a689. Thank you for the review - made the code much better.