twmr / gerrit.el

gerrit integration in emacs
43 stars 6 forks source link

Number of gerrit accounts limit (=500) #8

Closed twmr closed 9 months ago

twmr commented 4 years ago

The GET /accounts/ endpoint returns a max number of 500 accounts. If the gerrit server has more than 500 accounts registered, then the client needs to call the /accounts endpoints multiple times.

See https://gerrit-review.googlesource.com/Documentation/rest-api-accounts.html

gerrit.el is currently limited to 500 accounts, but the following patch should fix that. The problem is that querying a couple of thousand accounts is very slow.

(defun gerrit-rest--get-gerrit-accounts ()
  "Return an alist of all active gerrit users."
  (interactive)
  (condition-case nil
      (let ((continue t)
            (start-idx 0)
            (accounts '()))
        (while continue
          (let ((response
                 ;; see https://gerrit-review.googlesource.com/Documentation/rest-api-accounts.html
                 ;; and https://gerrit-review.googlesource.com/Documentation/user-search-accounts.html#_search_operators
                 (gerrit-rest-sync "GET" nil (format "/accounts/?q=is:active&o=DETAILS&S=%s" start-idx))))
            (setq accounts (append
                           accounts
                           (mapcar (lambda (account-info)
                                     (cons (alist-get '_account_id account-info)
                                           (alist-get 'username account-info)))
                                   response)))
            (setq start-idx (+ start-idx (length response)))
            ;; (message "start: %s" start-idx)
            (setq continue (alist-get '_more_accounts (car (last response))))))
            accounts)
    (error '())))

For tests the review.gerrithub.org gerrit instance can be used, which has between 10000 and 20000 accounts.

For comparison here is a simple shell script that uses curl to download the accounts. TODO output some numbers

#!/bin/bash
for i in `seq 0 500 10000`; do
  echo $i;
  curl --silent -o out/$i.out -n https://review.gerrithub.io/a/accounts/?q=is:active\&o=DETAILS\&S=$i
done
twmr commented 9 months ago

Here is an updated version that uses gerrit-rest-sync-v2

(defun gerrit-rest--get-gerrit-accounts2 ()
  "Return an alist of all active gerrit users."
  (interactive)
  (condition-case nil
      (let ((continue t)
            (start-idx 0)
            (accounts '()))
        (while continue
          (let ((response
                 ;; see https://gerrit-review.googlesource.com/Documentation/rest-api-accounts.html
                 ;; and https://gerrit-review.googlesource.com/Documentation/user-search-accounts.html#_search_operators
                 (gerrit-rest-sync-v2 "GET" "/accounts/"
                                      :params `(("q" "is:active")
                                                ("o" "DETAILS")
                                                ("S" ,start-idx)))))
            (setq accounts (append
                            accounts
                            (mapcar (lambda (account-info)
                                      (cons (alist-get '_account_id account-info)
                                            (alist-get 'username account-info)))
                                    response)))
            (setq start-idx (+ start-idx (length response)))
            (setq continue (alist-get '_more_accounts (car (last response))))
            ;; (message "start: %s, continue %s" start-idx continue)
            ))
        accounts)
    (error '())))