Can't get multi-line suggestion though I have a pro account

[allan-simon]

It successfully prompt me one-liner snippet but it does not seem to prompt for multi-line

Is there anyway I can provide you with more debugging information ?

[allan-simon]

is it related to https://github.com/codota/TabNine/issues/558 ?

https://github.com/neoclide/coc-tabnine/issues/81

[tzachar]

I have no idea. If you can supply more info, I would gladly help. Anyway, a PR is always welcomed.

[allan-simon]

I have no idea. If you can supply more info, I would gladly help.

that was exactly why I asked

Is there anyway I can provide you with more debugging information ?

i.e if you tell me what you need , I can tell it to you

• Is there somewhere in your code that I should add some debugging information to see where suggestions from tabnine are returned ?
• Is it the content of a log file that you need ?
• Is there a verbose mode in which I should run the plugin ?

[allan-simon]

Anyway, a PR is always welcomed.

I have no problem on that , but a little pinch to get started would help :)

[tzachar]

The only thing I know about is enabling the multiline option through the Hub. According to https://github.com/codota/TabNine/blob/master/HowToWriteAClient.md, there is nothing else I can add. Try contacting TabNine's team. If you are a paid user, contact their support.

[allan-simon]

Thanks, don't get me wrong I know it's an opensource project and that you owe me nothing :)

My question was more, as you said with your link, the API seems quite simple, so I was more asking for quick pointer, where is the place in the code where you get the raw result from Tabnine so I can add some debug about the raw result so I can see if either:

1. the Tabnine API itself does not return multi-line -> I will contact support as clearly the problem will be on their side
2. The tabnine API returns multi-line but it disappear somehow -> I will know it's somewhere in this plugin or cmp itself

just this little information will get me on track and from there I stop bothering you, promise :)

[tzachar]

https://github.com/tzachar/cmp-tabnine/blob/cfd93698c1ada2c0f493e48f0df19930d6cdc218/lua/cmp_tabnine/source.lua#L293

[allan-simon]

thanks a lot :)

[allan-simon]

I see that this plugin does the request a bit differently than the vscode plugin

https://github.com/tzachar/cmp-tabnine/blob/cfd93698c1ada2c0f493e48f0df19930d6cdc218/lua/cmp_tabnine/source.lua#L205-L217

(with an hardcoded API version to 3.3.0 ) while the vscode does this

https://github.com/codota/tabnine-vscode/blob/master/src/runCompletion.ts#L21-L34

so with some additional/different parameter

https://github.com/codota/tabnine-vscode/blob/master/src/binary/InnerBinary.ts#L42-L47

with API_VERSION being equal to 4.4.71

could this be the reason ? ( I'm trying to replicate the request on my side but I'm far from being a nvim plugin expert so I'm making slow progress, I'm posting that here if that ring a bell for you )

[allan-simon]

using strace here's what I can find vscode extension is sending

{
   "version":"4.4.71",
   "request":{
     "Autocomplete":{
          "filename":"Untitled-3.py",
          "before":"# check if  number is prime\ndef isPrime", 
          "after":"",
          "region_includes_beginning":true,
          "region_includes_end":true,
          "max_num_results":5,
          "offset":38,
          "line":1,
          "character":10,
          "indentation_size":4
      }
   }
}

[allan-simon]

here's a beginning of change that at least is considered valid by Tabnine

  local req = {}      
  req.version = '4.4.71'
  req.request = {
    Autocomplete = {
      before = before,
      after = after,
      region_includes_beginning = region_includes_beginning,
      region_includes_end = region_includes_end,
      -- filename = fn["expand"]("%:p"),
      filename = vim.uri_from_bufnr(0):gsub('file://', ''),
      max_num_results = conf:get('max_num_results'),
      correlation_id = ctx.context.id,
      line = cursor.line,    
      offset = #before + 1,
      character = cursor.col,
      indentation_size = 4,
    },
  }

[tzachar]

Most of your additions (other than bumping the current version) are not documented in their public interface.

Anyway, I tried your suggestions and did not get anything. Will try more tomorrow.

[allan-simon]

so I forget to say but I tested locally with vscode and their plugin, and I got multiline autocompletion, so it's not a problem with my account or the local tabnine client , but certainly a difference between how this plugin communicate with the local process

hence why I'm trying to make both as close as possible until I hit the one important difference :)

[allan-simon]

I'm making progress

with these changes

@@ -193,7 +194,8 @@ function Source._do_complete(self, ctx)
     region_includes_end = true
   end

-  local lines_before = api.nvim_buf_get_lines(0, cursor.line - max_lines, cursor.line - 1, false)
+  local lines_before = api.nvim_buf_get_lines(0, cursor.line - max_lines, cursor.line, false)
   table.insert(lines_before, cur_line_before)
   local before = table.concat(lines_before, '\n')

@@ -202,7 +204,7 @@ function Source._do_complete(self, ctx)
   local after = table.concat(lines_after, '\n')

   local req = {}
-  req.version = '3.3.0'
+  req.version = '4.4.71'
   req.request = {
     Autocomplete = {
       before = before,
@@ -213,8 +215,13 @@ function Source._do_complete(self, ctx)
       filename = vim.uri_from_bufnr(0):gsub('file://', ''),
       max_num_results = conf:get('max_num_results'),
       correlation_id = ctx.context.id,
+      line = cursor.line,
+      offset = #before + 1,
+      character = cursor.col,
+      indentation_size = 4,
     },
   }

if I dump the request I got something like this :

{"version": "4.4.71", "request": {"Autocomplete": {"before": "# check if number is prime\\ndef isPrime", "region_includes_beginning": true, "filename": "/tmp/something.py", "max_num_results": 20, "offset": 39, "line": 1, "correlation_id": 293, "indentation_size": 4, "region_includes_end": true, "after": "", "character": 12}}}

and in nvim I see this

now if close vim, and that I launch the tabnine server directly

~/.local/share/nvim/plugged/cmp-tabnine/binaries/4.4.141/x86_64-unknown-linux-musl/TabNine

and I enter in stdin directly this :

{"version": "4.4.71", "request": {"Autocomplete": {"before": "# check if number is prime\\ndef isPrime(toto):\n", "region_includes_beginning": true, "filename": "/tmp/something.py", "max_num_results": 20, "offset": 39, "line": 1, "correlation_id": 293, "indentation_size": 4, "region_includes_end": true, "after": "", "character": 12}}}

(so everything the same except that i added (toto):\n

now I got :drum: :drum:

{
  "old_prefix": "",
  "results": [
    {
      "new_prefix": "#    if toto % 2 == 0:\n#        return 1\n#    else:\n#        return isPrime(toto // 2)\n",
      "old_suffix": "",
      "new_suffix": "",
      "kind": 15,
      "origin": "CLOUD2",
      "detail": "-18.741",
      "completion_kind": "Snippet",
      "is_cached": false
    },
    {
      "new_prefix": "#    if toto % 2 == 0:\n#        return 1\n#    else:\n#        return isPrime(toto // 2)\n",
      "old_suffix": "",
      "new_suffix": "",
      "kind": 15,
      "origin": "CLOUD2",
      "detail": "-20.658",
      "completion_kind": "Snippet",
      "is_cached": false
    }
  ],
  "user_message": [],
  "docs": [],
  "is_locked": false,
  "snippet_context": {
    "stop_reason": "stop_token",
    "generated_tokens": 60,
    "user_intent": "Comment",
    "intent_metadata": {
      "current_line_indentation": 0,
      "previous_line_indentation": 0,
      "triggered_after_character": ":"
    },
    "response_time_ms": 879,
    "is_cached": false,
    "context_len": 47
  }
}

multi-line completion :tada:

now I got to figure out how to trigger than from nvim

[allan-simon]

if I change my nvim config by

cmp.setup({
    snippet = {
      expand = function(args)
        require('snippy').expand_snippet(args.body) -- For `snippy` users.
      end,
    },
    sources = {
      {name = 'cmp_tabnine', keyword_length = 0 },  <------ keywrod_length = 0 to be added 
      {name = 'snippy' },
    },

then I can see the request being sent by my nvim and tabnine replying the above answers

BUT

the answer from tabnine does not have a correlation_id so this plugin can not get it :(

[allan-simon]

actually all my changes above (except my PR #70 ) are not needed per se, it already works with keyword_length = 0 (so keeping 3.3.0 as protocol and same arguments ) , it's just that the answer does not have a correlation_id so the response is discarded by this plugin

I've contacted tabnine about this , as the fact the correlation_id is very certainly a bug on their side

[allan-simon]

that will be all for today for me

if I do an ugly hack and replace Source.pending to only contains the very last job

i.e

-  self.pending[ctx.context.id] = { ctx = ctx, callback = callback, job = self.job }
+  self.pending = { ctx = ctx, callback = callback, job = self.job }

and

-  self.pending = {}
+  self.pending = nil
   self.job = fn.jobstart({ bin, '--client=cmp.vim' }, {
     on_stderr = nil,
     on_exit = function (j, c, _) self:on_exit(j, c) end,
@@ -291,21 +293,19 @@ function Source.on_stdout(self, data)
   for _, jd in ipairs(data) do
     if jd ~= nil and jd ~= '' and jd ~= 'null' then
       local response = (json_decode(jd) or {})
-      local id = response.correlation_id
       if response == nil then
         dump('TabNine: json decode error: ', jd)
       elseif (response.message or ''):find('http://127.0.0.1') then
         self.hub_url = response.message:match('.*(http://127.0.0.1.*)')
-      elseif id == nil then
-          -- ignore this message
-      elseif self.pending[id] == nil then
+      elseif self.pending == nil then
         dump('TabNine: unknown message: ', jd)
-      elseif self.pending[id].job ~= self.job then
+      elseif self.pending.job ~= self.job then
         -- a message from an old job. skip it
       else
-        local ctx = self.pending[id].ctx
-        local callback = self.pending[id].callback
-        self.pending[id] = nil
+        local ctx = self.pending.ctx
+        local callback = self.pending.callback
+        self.pending = nil

then multi-line works

[tzachar]

This is problematic, as it disregards the sync between cmp and tabnine, and you can get stale completion suggestions.

[allan-simon]

yes I know it's an ugly hack , hence the disclaimer, I don't mean at all for it to be merged (or a PR with these changes would have been opened) I was merely checking if the correlation_id issue was the only issue left , or if there was stuff "behind" that would also need to be corrected.

[tzachar]

I think I am also making some. In general, I did not need your hack, however I cannot find a way to consistently get multiline suggestions. Do you have a good starting point?

[allan-simon]

first it seems that some language get them and some not (i have multi-line with python but not with php for example)

with python I always get some when i do

# a comment explaining
| <---- cursor here

[tzachar]

Not consistent for me. Can you give a specific comment you are using?

[allan-simon]

the

# check if a number is SOMETHING

seems to give good result (prime, odd, even )

[otrebu]

Hello, sorry to write on a close issue, but is it definitely working for you now?

[allan-simon]

yes I got multi-line snippet now

[otrebu]

Ooh I am not getting them, the readme says about that should show in the documentation side, but they don't for me. It must be something to do with my configuration... But also I never see a ML hint. I do get multiline in VSCode.

[aemonge]

Same here, I'm not getting any multiline in the documentation.

[tzachar]

I think multiline suggestions are still a bit finicky with tabnine. Its not a problem with the plugin.

[otrebu]

I am not sure, every time I go back to VsCode they seem to work. I think it might be my setup, I will try to change it around.

[tzachar]

@otrebu can you set up a sample where you persistently get multi line suggestions?

[otrebu]

Thanks @tzachar you are very supportive, bare with me for a few days while I try to review my config ( I also want to try to set it you using lunarvim ) and then I will share example and config with you.

[tzachar]

Was not talking about config, rather about a code sample for which u get multi line suggestions in vscode.

[otrebu]

Yes I understand @tzachar, but I am thinking that also my configuration can affect the multiline suggestions so I was thinking to check and potentially share both 😄

[aemonge]

If you find it useful, this is my LunarVim configuration. The part for the tabNine is simple enough:

https://github.com/aemonge/lettuce/blob/master/lvim.config.lua#L345

[aemonge]

I've been asking TabNine directly, and @tzachar is 100% right it's on the API it self. They told me:

Yes, the advanced features you described are not available yet on neovim (Only on VSCode and Jetbrains). We apologize for the disappointment! You should follow up on our newsletter if we ever add it.

[otrebu]

Oh thanks @aemonge that is a bad news, the "ever add it" sounds quite a far possibility. Oh well, I was just considering cancelling my subscription and I can do that and put this as a reason.

[tzachar]

@aemonge I don't really understand what you are saying. I did manage to get multi line suggestions using this plugin on neovim. However, I did not manage to get them consistently.

[allan-simon]

@aemonge the answer from tab9 makes no sense, as they only expose a HTTP api, the fact it is supported after or not is up to the plugin maintainer, and as in the case of this repository, it'sw not maintained by them, so they can't really have a clue on wether t's implemented or not

after you have to know that not all languages have that support enabled on their side (for example you will not have it with PHP, but you will with python )

[aemonge]

Hi @allan-simon, I've also had a personal Zoom call with one of their product owner, and he insisted that the support of multi-line it's only for those editors. Though I do understand this is a bit weird, so if you manage to make the multi-line visible in the documentation, please let me know. I've stopped using Tab9 PRO for that reason

:bow: thanks in advance :bow:

[tzachar]

This is strange. The only way for T9 to know what is the editor is through the information supplied through the editor. When launching T9, we identify as cmp: see here

You can try to change that to identify as something else, maybe vscode like here

Do share your results :)

[nfwyst]

same here~

[otrebu]

I was pointed to https://github.com/codota/tabnine-nvim by a product manager from Tabnine, maybe it can help. It is still in alpha version.

[aemonge]

I was about to say the same thing @otrebu, it seems like they have prioritized support for other editors. If any of you, dear mates, have a successful multi-line integration with cpm-tab9, please let us know how good/bad/ok the experience is :)

[aemonge]

For closure:

require('tabnine').setup({
  disable_auto_comment=true,
  accept_keymap=nil,
  disable_suggestion=true,
  debounce_ms = 300,
  suggestion_color = {gui = "#808080", cterm = 244},
  plugins = {
    cmp= true
  }
})

As mentioned by eloycoto in this discussion: https://github.com/codota/tabnine-nvim/issues/6

[tzachar]

see https://github.com/codota/tabnine-nvim/issues/6#issuecomment-1364655503 once we get a fix from tabnine, you should start getting multiline suggestions.

[nfwyst]

@tzachar still not working

[tzachar]

@nfwyst did you enable cloud and beta versions?

[nfwyst]

@tzachar yes

[tzachar]

Well, it currently works for me... can you add more information?