Weird, Irrelevant Auto-Complete Suggestions (Escape Characters, etc)

[ejmg]

Description :octocat:

Weird and "irrelevant" auto-complete suggestions from company-mode in latex-mode, magit commit message buffers, and various (sporadically?) others. Notably, these completion candidates include escape sequences as will be seen below.

While I have observed this behavior in other modes, it was never common enough for me to care while latex-mode has been very consistent in displaying it during general editing/work on a .tex file. Additionally, magit-mode generally does not display this behavior but within commit comment buffers this behavior is consistently seen as well. Finally, for latex-mode, the probability of irregular completion suggestions is extremely common whereas within a commit buffer the behavior occurs only moderately.

I have noticed this behavior for some time now and have decided to finally report it as I don't think this is something of my doing.

(FYI: Reporting from within spacemacs did not work in terms of successfully sending the issue to Github, I had to copy & paste this from the buffer I wrote it in case that's a tangential but real concern)

Reproduction guide :beetle:

• Start Emacs
• Open .tex file
  • Alternatively, open a commit message buffer for magit-mode
• Enter almost any character
• Yields weird completion suggestions by company-mode

Observed behaviour: :eyes: :broken_heart:

Example from literally within the REPORT_SPACEMACS_ISSUE buffer I am filling out for this issue:

Entering hw yields the completion option bc3��O�J (no company menu, inline recommendation).

In latex-mode, entering h yields the following recommendation, amongst others: hA̲DbO�T�բ���gS�.

Another example from a latex file:

Two from a single magit commit buffer:

Expected behaviour: :heart: :smile: Sane, predictable completion options only. As of now, completion options include both expected and the completely unexpected as described above.

System Info :computer:

• OS: gnu/linux
• Emacs: 25.1.1
• Spacemacs: 0.200.7
• Spacemacs branch: master (rev. c2774bc)
• Graphic display: t
• Distribution: spacemacs
• Editing style: emacs
• Completion: helm
• Layers:

  (octave ruby helm
      (auto-completion :variables auto-completion-enable-help-tooltip t auto-completion-enable-snippets-in-popup t auto-completion-enable-sort-by-usage t)
      gtags spell-checking syntax-checking csv python
      (c-c++ :variables c-c++-enable-clang-support t c-c++-default-mode-for-headers 'c++-mode)
      rust
      (common-lisp :variables common-lisp-style-default 'sbcl)
      javascript better-defaults emacs-lisp git markdown html java org latex bibtex yaml haskell clojure scala nginx pdf-tools ipython-notebook
      (erc :variables erc-server-list
           '((REDACTED)))
      (shell :variables shell-file-name "bash" shell-default-height 30 shell-default-position 'bottom)
      version-control)

• System configuration features: XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11

Backtrace :paw_prints:

N/A unless requested.

[CeleritasCelery]

Use company-diag in an affected mode to see what company backend is at fault. Also company is not enabled in magit buffers so I am puzzled how you are getting autocompletion popups there.

[ejmg]

@CeleritasCelery Thanks for the response. Here's what company-diag gives for an open .tex file.

Emacs 25.1.1 (x86_64-unknown-linux-gnu) of 2016-12-26 on REDACTED
Company 0.9.4

company-backends: ((company-auctex-macros company-auctex-symbols company-auctex-environments :with company-yasnippet)
 (company-auctex-bibs :with company-yasnippet)
 (company-auctex-labels :with company-yasnippet)
 (company-dabbrev-code company-gtags company-etags company-keywords :with company-yasnippet)
 (company-files :with company-yasnippet)
 (company-dabbrev :with company-yasnippet))

Used backend: (company-dabbrev :with company-yasnippet)

Major mode: latex-mode
Prefix: ""
Completions:

Looking at the completion options, I now at least understand why I'm getting these options. They're all there and the list is huge. I don't know if these are candidates or options previously used/completed, but here is a very small sampling which conforms with the examples in my initial post:

"C\302\355\221"
  "\361\270\352e\226"
  "I\342\245\216\362\301\3657"
  "V\377\243Z\264l"
  "\261\271\335c\363\307\330t\235\265t\237"
  "\335eK\267\342"
  "p\201\265\376\256"
  "R5mGL"
  "\361\337\374\350c\270\327"
  "\311\210\240\243\262"
  "o\357\372\351\275"
  "7\335\206\233\260\201\266\272\305\315"
  "\300Uh\252z"
  "\216\340W\370"
  "\364A\362a"
  "\274s\3174B"
  "\227\303\377\317s\255\230\236\276"

Between these trash suggestions are, of course, the sane and desired ones.

[CeleritasCelery]

Do you get the same suggestions when you call dabbrev-completetion or dabbrev-expand?

[Alexander-Miller]

Looks like company-dabbrev is picking its completions from some buffer thats filled with trash. Try cleaning up your buffer list, those weird numbers remind me of what you see when you open a binary in a text editor. You can also try different values for company-dabbrev-other-buffers.

[ejmg]

@CeleritasCelery Assuming I've entered at least one character, like 5, dabbrev-expand will do nothing but dabbrev-completion will provide a list of normal suggestions, like "normal" numbers, not the endless list of escape sequence characters above.

The behavior is the same whether company mode is enabled or not (not sure that matters but wanted to control for it).

---

@Alexander-Miller You may be on to something. I don't keep my buffers polluted with a million old things still open, but I closed out my emacsclient session, restarted a session, and opened a tex file immediately. All my suggestions were normal and company-diag reflected that.

Prior to closing out all my buffers and doing the above, I do want to note that I had not knowingly ever opened a binary file and that of all my buffers open, none were binary in nature. Assuming I did accidentally open a binary file (as the causal source), why is company/emacs holding on to those completions? Is there a way to preemptively stop this behavior?

Thank you both for your input up to this point :slightly_smiling_face:

[Alexander-Miller]

I did not mean that you literally opened a binary file, just something thats not meant to be opened in a text editor - a binary, a picture, a video, something like that. In fact those backslash separated numbers look suspiciously like elisp bytecode.

There's a couple things you can do:

• Adjust the value of company-backends to remove company-dabbrev, globally or locally. Its fairly far down in the backend order, so you wont lose that much.
• Adjust the value of company-dabbrev-other-buffers to only look for completions in the current buffer.
• Find out where those weird completions come from and kill those buffers early.
• Find out where those weird completions come from and add their name to company-dabbrev-ignore-buffers.

BTW since company-dabbrev is so far down in company-backends the only times you should ever see those weird completions is when no one else has anything to offer, especially those backends dedicated to latex.

This is all assuming were digging in the right direction. Judging from the prefix and completions values you did not use company-diag correctly. You are supposed to call it with the cursor in the same spot where youre getting undesired completions. Another thing you can do is just manually call a company backend and see which completions youre offered.

[ejmg]

Okay, so update. @Alexander-Miller was right more or less with respect to picking up weird completion from compiled files. I took it as obvious within the discussion that if I was using spacemacs for .tex files, I must also be using it for compiling and viewing the respective .pdf's they made. I'm not sure if just by compiling .tex files with spacemacs fills company-mode with the junk suggestions or if you must open the file directly, I did both.

Doing some greping/ag searches for the completion strings with character strings long enough to yield a result gave me output from .pdf itself for the really nasty completions I showed above, along with some results from .fdb_latexmk for generically long character strings without escape characters in them. These were the two main culprits for junk suggestions.

This is my hack of a solution around this and I went ahead and added all the intermediate files that latex compilation generates. I don't know how to append a list of values to another list for variables with e-lisp so I would appreciate help with cleaning it up:

   (push ".fbd_latexmk" company-files-exclusions)
   (push ".aux" company-files-exclusions)
   (push ".log" company-files-exclusions)
   (push ".pdf" company-files-exclusions)
   (push ".bbl" company-files-exclusions)
   (push ".bcf" company-files-exclusions)
   (push ".gz" company-files-exclusions)
   (push ".blg" company-files-exclusions)
   (push ".fls" company-files-exclusions)

After restarting spacemacs, I visited a .tex file, compiled it, and viewed the resulting .pdf from within spacemacs. Going back to .tex and trying various characters yielded no trash suggestions like before.

Thank you both for your help figuring this out.

If this is satisfactory as an answer to my problem, feel free to close it (though I would appreciate the advice for cleaning up my hack!).

[Alexander-Miller]

Your answer is pretty much the right way to go about this, so we can close this. At least you can - I dont have that power, I assume only people who own the project or opened the issue in the first place can close it. As for feedback:

push is the right way to add an item to a list. For a bit extra control - that you dont need - theres also add-to-list. To append a list of values to a list you use append:

(setq foo (append '(1 2 3) foo))`

You can also do a loop:

(dolist (extension '(".gz" ".blg" etc.))
    (push extension company-files-exclusions))

[peterwvj]

Manually maintaining company-files-exclusions doesn't work well if you have a lot of buffers open. Generally, it's difficult to predict which files and buffers will contain garbage suggestions. A solution that works better for me is to limit company-dabbrev to only search buffers with the same major mode:

(setq company-dabbrev-other-buffers t)