company-mode poor performance with large completion candidates

[liujoey]

I searched every where but couldn't find a solution, so I guess it's best to open an issue here. What I experienced is better provided by an example:

import org.testng.Assert;

public class sometest {
  Assert.assertEquals();
}

The issue is very general, here is just an example. The Assert class has about more than 30 overrides of assertEquals. When I type Assert. the company-mode pops up a list and then I start to type assert. While I'm typing, the performance of typing drops down to like a 'Power Point Slide'. Every key I type will delay 2-3 seconds, looks like the company is updating the completion list every time.

When I run profiler, lot of CPU time is spent by eclim--accepted-p:

- company-post-command                                           1059  18%
 - funcall                                                       1059  18%
  - #<compiled 0xd9c7cf>                                         1059  18%
   - company--perform                                             709  12%
    - company--continue                                           598  10%
     - company-calculate-candidates                               594  10%
      - company--postprocess-candidates                           594  10%
       - company--transform-candidates                            594  10%
        - company-sort-by-occurrence                              594  10%
         - cl-delete-if                                           594  10%
          - apply                                                 594  10%
           - cl-delete                                            588  10%
            - #<compiled 0x1604393>                               579  10%
             - funcall                                            571   9%
              - #<compiled 0x9345c7>                              564   9%
               - company--occurrence-predicate                    537   9%
                - company-call-backend                            537   9%
                 - company--force-sync                            537   9%
                  - apply                                         537   9%
                   - company-call-backend-raw                     536   9%
                    - funcall                                     536   9%
                     - #<compiled 0x158d599>                      536   9%
                      - apply                                     536   9%
                       - company-emacs-eclim                      534   9%
                        - eclim--accepted-p                       474   8%
                         + eclim--file-managed-p                  471   8%
                           eclim--accepted-filename-p                  3   0%
                        + eclim-completion-start                   50   0%

So the question will be, do we need to query eclim every time when we narrow down the list? Since the first query already get a full list of all candidates, so should following actions just reuse what ever in memory and avoid to communicate with eclim?

PS: I'm completely new to lisp programming, I might be wrong from any perspective.

EDIT: By removing the function call of eclim--accepted-p, all performance issues I've experienced so far are removed. This function call was first introduced via #163

[kleewho]

@liujoey Thank you for your detailed bug report. I'll look into this.

[Kethku]

You made my day. I had been using emacs eclim because it was the only useful java tooling in emacs, but I had always been frustrated with the slow performance. After removing the eclim--accepted-p call and recompiling it runs smooth as butter. I have no idea what eclim--accepted-p actually does, and honestly I don't care. The experience is SOO much better now.

[liujoey]

this function will check if the file you are editing is actually managed by a eclim project, which I think is not necessary. Inside the function, it will eventually trigger a file save action which leads to the slowness.

[kleewho]

@liujoey Ok, so I don't want to drop completely this code but I'm able to make it faster. I'll shortly prepare some fix for that and then you can check whether it's fine.

[kleewho]

As a proof. Before:

Function name                                      Call count  Elapsed time  Average time
eclim--execute-command-internal                    7           0.263651921   0.0376645601
eclim--completion-candidates                       1           0.163290677   0.163290677
eclim--complete                                    1           0.162877331   0.162877331
eclim--call-process                                8           0.147610166   0.0184512707
eclim/java-src-update                              1           0.129725927   0.129725927
eclim--accepted-p                                  5           0.101037635   0.020207527
eclim--file-managed-p                              5           0.1008748670  0.0201749734
eclim-project-name                                 9           0.100849282   0.0112054757

After:

Function name                                      Call count  Elapsed time  Average time
eclim--execute-command-internal                    2           0.13716337    0.068581685
eclim--completion-candidates                       1           0.119123926   0.119123926
eclim--complete                                    1           0.118724757   0.118724757
eclim/java-src-update                              1           0.083501006   0.083501006
eclim--call-process                                3           0.0757254689  0.0252418229
eclim--after-save-hook                             1           0.022646935   0.022646935
eclim-problems-highlight                           7           0.019787082   0.0028267260
eclim--accepted-p                                  14          0.0193173419  0.0013798101
eclim--file-managed-p                              10          0.0188070310  0.0018807031
eclim-project-name                                 22          0.0187635599  0.0008528890

eclim-project-name now do not call eclim if not needed and therefore whole eclim--accepted-p is faster.

Check this PR https://github.com/senny/emacs-eclim/pull/237

[liujoey]

now it's not completing at all. I'm using company mode if it is matter.

BTW: I didn't drop the entire function, I just removed the call from company mode. This line

[kleewho]

Yes I know that, but I think the point raised in #163 is valid so I'd like to keep the call in here as well. I don't expect it to break any completion, but I'll check it

[kleewho]

Can you update whole eclim and then check once again?

[liujoey]

does eclim version matter? I'm running Eclipse Luna and eclim 2.4.1.

I've updated the whole plugin, but still not getting any completion.

When I tried to debug the eclim-project-name function, it shows edebug-syntax-error: Invalid read syntax: "Expected lambda expression"

[kleewho]

Can you update whole eclim and then check once again?

I was thinking about emacs-eclim only. Sorry for confusion

The only function that have this "Expected lambda expression" in edebug is this:

(defun edebug-match-lambda-expr (cursor)
  ;; The expression must be a function.
  ;; This will match any list form that begins with a symbol
  ;; that has an edebug-form-spec beginning with &define.  In
  ;; practice, only lambda expressions should be used.
  ;; I could add a &lambda specification to avoid confusion.

How do you try to debug it?

[liujoey]

I was trying to debug function eclim-project-name, from my little emacs knowledge, before I set break point in a elisp function, I need to eval the function with instruments by pressing C-u C-M-x, but when I did this on function eclim-project-name, it showed me above error message.

Then I tried to debug function company-emacs-eclim, with the keyC-u C-M-x, it successfully setup the instrument and let me able to trace into it, and the return value of (eclim--accepted-p (buffer-file-name)) is nil.

Please excuse me about my bad emacs knowledge, really have no time to dive into elisp at this moment, project dead line right ahead you know :) What I'm trying to do is providing you information as much as I can. Thank you.