gocode Panic on Windows

[Hanan-Natan]

I'm getting the error [deoplete] gocode panicked on Windows. I have the following environment:

OS: Windows 7 x64 NVIM v0.2.3-527 Latest installation of deoplete, deoplete-go and gocode.

When running gocode in debug mode i see the error:

2018/02/01 09:20:27 Go project path: hashMePlease
2018/02/01 09:20:27 Got autocompletion request for 'c:\Users\hanann\go\src\hashMePlease\main.go'
2018/02/01 09:20:27 Cursor at: 82
2018/02/01 09:20:27 ERROR! Cursor is outside of the boundaries of the buffer, this is most likely a text editor plugin bug. Text editor is responsible for passing the correct cursor position to gocode.

Here's my init.vim file:

 let g:python3_host_prog='C:/Tools/Neovim/MyEnvs/neovim3/Scripts/python.exe'
let g:python_host_prog='C:/Tools/Neovim/MyEnvs/neovim/Scripts/python.exe'
call plug#begin('~/AppData/Local/nvim/plugged')
    Plug 'Shougo/deoplete.nvim', { 'do': ':UpdateRemotePlugins' }
    Plug 'zchee/deoplete-go', { 'do': 'make'}
call plug#end()
let g:deoplete#enable_at_startup = 1
let g:deoplete#sources#go#gocode_binary = $GOPATH.'/bin/gocode.exe'

Here's the file I'm getting error with:

package main

import "fmt"

func main() {
    fmt.Println("Starting...");
}

The same error occur on Windows 10 x64

[Shougo]

I don't know about it is Windows specific issue. Please test it in Linux environment. I cannot reproduce the problem in Ubuntu 16.04.

[Shougo]

Please upload the reproduce instructions.

[Shougo]

I cannot support Windows specific behavior. You need to debug the problem.

Hint:

https://github.com/zchee/deoplete-go/blob/master/rplugin/python3/deoplete/sources/deoplete_go.py#L173

2018/02/01 09:20:27 Cursor at: 82

The cursor position is calculated by self.get_cursor_offset() in L 174. It uses utf-8 codec. I think it is broken in Windows. Your Vim or gocode does not use utf-8. Please check the encodings.

[Shougo]

I think it should be:

   def get_cursor_offset(self, context):
        line = self.vim.current.window.cursor[0]
        column = context['complete_position']

        return self.vim.call('line2byte', line) + \
            charpos2bytepos(context['encoding'], context['input'][: column], column) - 1

[Shougo]

Ah, it seems \r and \r\n problem.

get_cursor_offset() seems count \r\n. But gocode does not count \r. So bytes offset will be broken.

[Shougo]

@Hanan-Natan Please upload the accurate file. Gist is better.

[Shougo]

@Hanan-Natan Please test below patch. I think it should work.

diff --git a/rplugin/python3/deoplete/sources/deoplete_go.py b/rplugin/python3/deoplete/sources/deoplete_go.py
index e382827..e84f080 100644
--- a/rplugin/python3/deoplete/sources/deoplete_go.py
+++ b/rplugin/python3/deoplete/sources/deoplete_go.py
@@ -234,9 +234,13 @@ class Source(Base):
     def get_cursor_offset(self, context):
         line = self.vim.current.window.cursor[0]
         column = context['complete_position']
-
-        return self.vim.call('line2byte', line) + \
-            charpos2bytepos('utf-8', context['input'][: column], column) - 1
+        count = self.vim.call('line2byte', line)
+        if self.vim.current.buffer.options['fileformat'] == 'dos':
+            # Note: line2byte() counts "\r\n" in DOS format.  It must be "\n"
+            # in gocode.
+            count -= line
+        return count + charpos2bytepos(
+            'utf-8', context['input'][: column], column) - 1

     def parse_import_package(self, buffer):
         start = 0

[Hanan-Natan]

Indeed it seems like you resolved the cursor issue. There's a different issue however.

2018/02/02 08:26:46 ======================================================= 2018/02/02 08:26:47 Go project path: hello 2018/02/02 08:26:47 Got autocompletion request for 'C:\dev\GoProjects\src\hello\hello.go' 2018/02/02 08:26:47 Cursor at: 76 2018/02/02 08:26:47 ------------------------------------------------------- package main import "fmt" func main() { fmt.Printf("hello, world\n") fmt#. } 2018/02/02 08:26:47 ------------------------------------------------------- 2018/02/02 08:26:47 Found "fmt" at "C:\dev\Go\pkg\windows_amd64\fmt.a" 2018/02/02 08:26:47 Error parsing input file (inner block): 2018/02/02 08:26:47 5:6: expected statement, found '.' 2018/02/02 08:26:47 6:2: expected ';', found 'EOF' 2018/02/02 08:26:47 6:2: expected '}', found 'EOF' 2018/02/02 08:26:47 Offset: 3 2018/02/02 08:26:47 Number of candidates found: 1 2018/02/02 08:26:47 Candidates are: 2018/02/02 08:26:47 package fmt 2018/02/02 08:26:47 =======================================================

[Shougo]

I don't know why the error is occurred.

[Shougo]

Please use it instead.

diff --git a/rplugin/python3/deoplete/sources/deoplete_go.py b/rplugin/python3/deoplete/sources/deoplete_go.py
index e382827..4480b61 100644
--- a/rplugin/python3/deoplete/sources/deoplete_go.py
+++ b/rplugin/python3/deoplete/sources/deoplete_go.py
@@ -234,9 +234,13 @@ class Source(Base):
     def get_cursor_offset(self, context):
         line = self.vim.current.window.cursor[0]
         column = context['complete_position']
-
-        return self.vim.call('line2byte', line) + \
-            charpos2bytepos('utf-8', context['input'][: column], column) - 1
+        count = self.vim.call('line2byte', line)
+        if self.vim.current.buffer.options['fileformat'] == 'dos':
+            # Note: line2byte() counts "\r\n" in DOS format.  It must be "\n"
+            # in gocode.
+            count -= line - 1
+        return count + charpos2bytepos(
+            'utf-8', context['input'][: column], column) - 1

     def parse_import_package(self, buffer):
         start = 0

[Hanan-Natan]

Now i can get completions for "fmt" package (which is already in the file) but when i add another import (i.e. "os") i get the following error from deoplete:

At the debug of gocode i can see the candidates (there's a parsing error however):

2018/02/02 09:19:34 =======================================================
2018/02/02 09:19:34 Go project path: hello
2018/02/02 09:19:34 Got autocompletion request for 'C:\dev\GoProjects\src\hello\hello.go'
2018/02/02 09:19:34 Cursor at: 88
2018/02/02 09:19:34 -------------------------------------------------------
package main
import "fmt"
import "os"
func main() {
fmt.Printf("hello, world\n")
os.#
}
2018/02/02 09:19:34 -------------------------------------------------------
2018/02/02 09:19:34 Found "fmt" at "C:\dev\Go\pkg\windows_amd64\fmt.a"
2018/02/02 09:19:34 Found "os" at "C:\dev\Go\pkg\windows_amd64\os.a"
2018/02/02 09:19:34 Error parsing input file (inner block):
2018/02/02 09:19:34 5:5: expected selector or type assertion, found ';'
2018/02/02 09:19:34 extracted expression tokens: os
2018/02/02 09:19:34 Offset: 0
2018/02/02 09:19:34 Number of candidates found: 99
2018/02/02 09:19:34 Candidates are:
2018/02/02 09:19:34 const DevNull
2018/02/02 09:19:34 const ModeAppend
2018/02/02 09:19:34 const ModeCharDevice

[Shougo]

gocode seems returned broken data.

[Shougo]

Please use this:

diff --git a/rplugin/python3/deoplete/sources/deoplete_go.py b/rplugin/python3/deoplete/sources/deoplete_go.py
index e382827..ecaaf38 100644
--- a/rplugin/python3/deoplete/sources/deoplete_go.py
+++ b/rplugin/python3/deoplete/sources/deoplete_go.py
@@ -229,14 +229,23 @@ class Source(Base):
             '\n'.join(buffer).encode()
         )

-        return loads(stdout_data.decode())
+        result = []
+        try:
+            result = loads(stdout_data.decode())
+        except Exception as e:
+            pass
+        return result

     def get_cursor_offset(self, context):
         line = self.vim.current.window.cursor[0]
         column = context['complete_position']
-
-        return self.vim.call('line2byte', line) + \
-            charpos2bytepos('utf-8', context['input'][: column], column) - 1
+        count = self.vim.call('line2byte', line)
+        if self.vim.current.buffer.options['fileformat'] == 'dos':
+            # Note: line2byte() counts "\r\n" in DOS format.  It must be "\n"
+            # in gocode.
+            count -= line - 1
+        return count + charpos2bytepos(
+            'utf-8', context['input'][: column], column) - 1

     def parse_import_package(self, buffer):
         start = 0

[Hanan-Natan]

This time no errors but no completions either (except for "fmt"). gocode log is the same (it finds candidates)

[Shougo]

This time no errors but no completions either (except for "fmt"). gocode log is the same (it finds candidates)

gocode result is broken. Please check it.

[Shougo]

You can check the result.

diff --git a/rplugin/python3/deoplete/sources/deoplete_go.py b/rplugin/python3/deoplete/sources/deoplete_go.py
index e382827..5c94297 100644
--- a/rplugin/python3/deoplete/sources/deoplete_go.py
+++ b/rplugin/python3/deoplete/sources/deoplete_go.py
@@ -229,14 +229,24 @@ class Source(Base):
             '\n'.join(buffer).encode()
         )

-        return loads(stdout_data.decode())
+        result = []
+        try:
+            result = loads(stdout_data.decode())
+        except Exception as e:
+            error(self.vim, 'gocode decode error')
+            error(self.vim, stdout_data.decode())
+        return result

     def get_cursor_offset(self, context):
         line = self.vim.current.window.cursor[0]
         column = context['complete_position']
-
-        return self.vim.call('line2byte', line) + \
-            charpos2bytepos('utf-8', context['input'][: column], column) - 1
+        count = self.vim.call('line2byte', line)
+        if self.vim.current.buffer.options['fileformat'] == 'dos':
+            # Note: line2byte() counts "\r\n" in DOS format.  It must be "\n"
+            # in gocode.
+            count -= line - 1
+        return count + charpos2bytepos(
+            'utf-8', context['input'][: column], column) - 1

     def parse_import_package(self, buffer):
         start = 0

[Hanan-Natan]

It seems like this change indeed solved the issue.

The other errors occurred because i ran gocode in debug mode gocode -s -debug. As soon as i stopped it everything works fine.

So to make sure it's working one should disable gocode debug mode output (which is the default) but if you can get it's status using gocode.exe set force-debug-output. It should be empty.

Thanks for your time an effort.

[Shougo]

OK.

I will send the PR later.