Cause plugin_host exit when rename file

[silva6]

When I rename a file, this plugin will cause plugin_host exit. This error is as follows.

INFO:EasyClangComplete.plugin.view_config: init completer based on libclang Traceback (most recent call last): File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 530, in on_activated_async callback.on_activated_async(v) File "/Users/lau/Library/Application Support/Sublime Text 3/Packages/EasyClangComplete/EasyClangComplete.py", line 137, in on_activated_async self.view_config_manager.load_for_view(view, settings) File "/Users/lau/Library/Application Support/Sublime Text 3/Packages/EasyClangComplete/plugin/view_config.py", line 341, in load_for_view res = self._cache[v_id].update_if_needed(view, settings) File "/Users/lau/Library/Application Support/Sublime Text 3/Packages/EasyClangComplete/plugin/view_config.py", line 89, in update_if_needed if ViewConfig.needs_reparse(view): File "/Users/lau/Library/Application Support/Sublime Text 3/Packages/EasyClangComplete/plugin/view_config.py", line 144, in needs_reparse if not File.is_unchanged(view.file_name()): File "/Users/lau/Library/Application Support/Sublime Text 3/Packages/EasyClangComplete/plugin/tools.py", line 216, in is_unchanged actual_mod_time = path.getmtime(file_path) File "./python3.3/genericpath.py", line 54, in getmtime FileNotFoundError: [Errno 2] No such file or directory: '/Users/lau/git/xxx/src/dao/FeedLikeDao.h' INFO:EasyClangComplete.plugin.view_config: init completer based on libclang error: plugin_host has exited unexpectedly, plugin functionality won't be available until Sublime Text has been restarted

I just change the name of "FeedLikeDao.h".

[niosus]

Do you rename it from within Sublime Text or from outside? I am renaming files regularly and it never resulted in such behavior.

[silva6]

Within sublime. I have test many times, and it cause error every time. When I remove this plugin, the error is disappear

[niosus]

Can you please do this again while having verbose setting of the plugin on? Please post the full log here. I cannot reproduce this, so I need to at least try to understand when this happens exactly. Otherwise it will be hard to fix.

[silva6]

INFO:EasyClangComplete.plugin.view_config: init completer based on libclang
DEBUG:EasyClangComplete.plugin.completion.lib_complete: using bundled cindex: EasyClangComplete.clang.cindex38
DEBUG:EasyClangComplete.plugin.flags_sources.cmake_file: [cmake]:[get]: for file /Users/lau/git/xxx/live_feeds/feed_like/src/cache/LikeCache.cpp
DEBUG:EasyClangComplete.plugin.flags_sources.cmake_file: [cmake]:[cached]: '/Users/lau/git/xxx/live_feeds/CMakeLists.txt'
DEBUG:EasyClangComplete.plugin.tools: searching 'CMakeLists.txt' from '/Users/lau/git/xxx/live_feeds/feed_like/src/cache' to '/Users/lau/git/xxx'
DEBUG:EasyClangComplete.plugin.tools: found 'CMakeLists.txt' file: /Users/lau/git/xxx/live_feeds/feed_like/CMakeLists.txt
DEBUG:EasyClangComplete.plugin.tools: skipping file '<EasyClangComplete.plugin.tools.File object at 0x11120a450>'. 
DEBUG:EasyClangComplete.plugin.tools: no line starts with: 'project'
DEBUG:EasyClangComplete.plugin.tools: found 'CMakeLists.txt' file: /Users/lau/git/xxx/live_feeds/CMakeLists.txt
DEBUG:EasyClangComplete.plugin.tools: found needed line: 'project(live_feeds)
'
DEBUG:EasyClangComplete.plugin.flags_sources.cmake_file: [cmake]:[current]: '/Users/lau/git/xxx/live_feeds/CMakeLists.txt'
DEBUG:EasyClangComplete.plugin.flags_sources.cmake_file: [cmake]: found cached CMakeLists.txt.
DEBUG:EasyClangComplete.plugin.flags_sources.cmake_file: [cmake]:[unchanged]: use existing db.
DEBUG:EasyClangComplete.plugin.flags_sources.compilation_db: [db]:[get]: for file /Users/lau/git/xxx/live_feeds/feed_like/src/cache/LikeCache
DEBUG:EasyClangComplete.plugin.flags_sources.compilation_db: [db]:[cached]: 'None'
DEBUG:EasyClangComplete.plugin.tools: searching 'compile_commands.json' from '/var/folders/5q/sk26ccz51bv8xqky5dc_t6_r0000gn/T/EasyClangComplete/cmake_builds/73c0665ad9fa08e156712dc9146e52b9' to '/'
DEBUG:EasyClangComplete.plugin.tools: found 'compile_commands.json' file: /var/folders/5q/sk26ccz51bv8xqky5dc_t6_r0000gn/T/EasyClangComplete/cmake_builds/73c0665ad9fa08e156712dc9146e52b9/compile_commands.json
DEBUG:EasyClangComplete.plugin.flags_sources.compilation_db: [db]:[current]: '/var/folders/5q/sk26ccz51bv8xqky5dc_t6_r0000gn/T/EasyClangComplete/cmake_builds/73c0665ad9fa08e156712dc9146e52b9/compile_commands.json'
DEBUG:EasyClangComplete.plugin.flags_sources.compilation_db: [db]: found cached compile_commands.json
DEBUG:EasyClangComplete.plugin.flags_sources.compilation_db: [db]:[load cached]
DEBUG:EasyClangComplete.plugin.flags_sources.compilation_db: [db]: return entry for 'all'.
DEBUG:EasyClangComplete.plugin.view_config: flags generated from 'CMakeLists.txt'.
DEBUG:EasyClangComplete.plugin.view_config: view config needs no update.
Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 530, in on_activated_async
    callback.on_activated_async(v)
  File "/Users/lau/Library/Application Support/Sublime Text 3/Packages/EasyClangComplete/EasyClangComplete.py", line 137, in on_activated_async
    self.view_config_manager.load_for_view(view, settings)
  File "/Users/lau/Library/Application Support/Sublime Text 3/Packages/EasyClangComplete/plugin/view_config.py", line 341, in load_for_view
    res = self._cache[v_id].update_if_needed(view, settings)
  File "/Users/lau/Library/Application Support/Sublime Text 3/Packages/EasyClangComplete/plugin/view_config.py", line 89, in update_if_needed
    if ViewConfig.needs_reparse(view):
  File "/Users/lau/Library/Application Support/Sublime Text 3/Packages/EasyClangComplete/plugin/view_config.py", line 144, in needs_reparse
    if not File.is_unchanged(view.file_name()):
  File "/Users/lau/Library/Application Support/Sublime Text 3/Packages/EasyClangComplete/plugin/tools.py", line 216, in is_unchanged
    actual_mod_time = path.getmtime(file_path)
  File "./python3.3/genericpath.py", line 54, in getmtime
FileNotFoundError: [Errno 2] No such file or directory: '/Users/lau/git/xxx/live_feeds/feed_like/src/cache/LikeCache.cpp'
DEBUG:EasyClangComplete.plugin.tools: file has valid syntax: `C++`
DEBUG:EasyClangComplete.plugin.tools: file has valid syntax: `C++`
DEBUG:EasyClangComplete.plugin.view_config: config exists for path: 113
DEBUG:EasyClangComplete.plugin.error_vis: no error regions for row: 22
DEBUG:EasyClangComplete.plugin.tools: file has valid syntax: `C++`
DEBUG:EasyClangComplete.plugin.tools: file has valid syntax: `C++`
DEBUG:EasyClangComplete.plugin.view_config: config exists for path: 113
DEBUG:EasyClangComplete.plugin.error_vis: no error regions for row: 22
DEBUG:EasyClangComplete.plugin.tools: file has valid syntax: `C++`
DEBUG:EasyClangComplete.plugin.tools: file has valid syntax: `C++`
DEBUG:EasyClangComplete.plugin.view_config: config exists for path: 113
DEBUG:EasyClangComplete.plugin.error_vis: no error regions for row: 18
DEBUG:EasyClangComplete.plugin.tools: file has valid syntax: `C++`
DEBUG:EasyClangComplete.EasyClangComplete: on_activated_async view id 113
DEBUG:EasyClangComplete.plugin.tools: file has valid syntax: `C++`
DEBUG:EasyClangComplete.plugin.view_config: config exists for path: 113
DEBUG:EasyClangComplete.plugin.tools: file has valid syntax: `C++`
INFO:EasyClangComplete.plugin.view_config: init completer based on libclang
DEBUG:EasyClangComplete.plugin.completion.lib_complete: using bundled cindex: EasyClangComplete.clang.cindex38
DEBUG:EasyClangComplete.plugin.flags_sources.cmake_file: [cmake]:[get]: for file /Users/lau/git/xxx/live_feeds/feed_like/src/cache/LikeCacheD.cpp
DEBUG:EasyClangComplete.plugin.flags_sources.cmake_file: [cmake]:[cached]: 'None'
DEBUG:EasyClangComplete.plugin.tools: searching 'CMakeLists.txt' from '/Users/lau/git/xxx/live_feeds/feed_like/src/cache' to '/Users/lau/git/xxx'
DEBUG:EasyClangComplete.plugin.tools: found 'CMakeLists.txt' file: /Users/lau/git/xxx/live_feeds/feed_like/CMakeLists.txt
DEBUG:EasyClangComplete.plugin.tools: skipping file '<EasyClangComplete.plugin.tools.File object at 0x11120a390>'. 
DEBUG:EasyClangComplete.plugin.tools: no line starts with: 'project'
DEBUG:EasyClangComplete.plugin.tools: found 'CMakeLists.txt' file: /Users/lau/git/xxx/live_feeds/CMakeLists.txt
DEBUG:EasyClangComplete.plugin.tools: found needed line: 'project(live_feeds)
'
DEBUG:EasyClangComplete.plugin.flags_sources.cmake_file: [cmake]:[current]: '/Users/lau/git/xxx/live_feeds/CMakeLists.txt'
DEBUG:EasyClangComplete.plugin.flags_sources.cmake_file: [cmake]: found cached CMakeLists.txt.
DEBUG:EasyClangComplete.plugin.flags_sources.cmake_file: [cmake]:[unchanged]: use existing db.
DEBUG:EasyClangComplete.plugin.flags_sources.compilation_db: [db]:[get]: for file /Users/lau/git/xxx/live_feeds/feed_like/src/cache/LikeCacheD
DEBUG:EasyClangComplete.plugin.flags_sources.compilation_db: [db]:[cached]: 'None'
DEBUG:EasyClangComplete.plugin.tools: searching 'compile_commands.json' from '/var/folders/5q/sk26ccz51bv8xqky5dc_t6_r0000gn/T/EasyClangComplete/cmake_builds/73c0665ad9fa08e156712dc9146e52b9' to '/'
DEBUG:EasyClangComplete.plugin.tools: found 'compile_commands.json' file: /var/folders/5q/sk26ccz51bv8xqky5dc_t6_r0000gn/T/EasyClangComplete/cmake_builds/73c0665ad9fa08e156712dc9146e52b9/compile_commands.json
DEBUG:EasyClangComplete.plugin.flags_sources.compilation_db: [db]:[current]: '/var/folders/5q/sk26ccz51bv8xqky5dc_t6_r0000gn/T/EasyClangComplete/cmake_builds/73c0665ad9fa08e156712dc9146e52b9/compile_commands.json'
DEBUG:EasyClangComplete.plugin.flags_sources.compilation_db: [db]: found cached compile_commands.json
DEBUG:EasyClangComplete.plugin.flags_sources.compilation_db: [db]:[load cached]
DEBUG:EasyClangComplete.plugin.flags_sources.compilation_db: [db]: return entry for 'all'.
DEBUG:EasyClangComplete.plugin.view_config: flags generated from 'CMakeLists.txt'.
DEBUG:EasyClangComplete.plugin.view_config: view config needs no update.
DEBUG:EasyClangComplete.plugin.tools: never seen file '/Users/lau/git/xxx/live_feeds/feed_like/src/cache/LikeCacheD.cpp' before. Updating.
DEBUG:EasyClangComplete.plugin.view_config: config updates existing completer.
DEBUG:EasyClangComplete.plugin.completion.lib_complete: view is 113
DEBUG:EasyClangComplete.plugin.completion.lib_complete: reparsing translation_unit for view 113
DEBUG:EasyClangComplete.plugin.tools: file has valid syntax: `C++`
DEBUG:EasyClangComplete.plugin.completion.lib_complete: reparsed in 0.0019769668579101562 seconds
error: plugin_host has exited unexpectedly, plugin functionality won't be available until Sublime Text has been restarted

[niosus]

Ok, it seems that a variable file_path does not reflect the actual path anymore. Even though I cannot reproduce this I guess I know how to fix it. I will fix in the coming days. Thanks for reporting.

[silva6]

Waiting for that. Thanks so much!

[silva6]

hi, niosus,

I add 3 line to toos.py as belows.

+        if not path.exists(file_path):
+            log.debug(" file '%s' does not exist.", file_path)
+            return False
         actual_mod_time = path.getmtime(file_path)

Unfortunately it seems has not fix the bug. The debug info look all right, but cause plugin_host exit.

DEBUG:EasyClangComplete.plugin.tools: file '/Users/lau/git/xxx/live_feeds/feed_like/src/cache/LikeCacheD.cpp' does not exist.
DEBUG:EasyClangComplete.plugin.view_config: config updates existing completer.
DEBUG:EasyClangComplete.plugin.completion.lib_complete: view is 141
DEBUG:EasyClangComplete.plugin.completion.lib_complete: reparsing translation_unit for view 141
DEBUG:EasyClangComplete.plugin.completion.lib_complete: reparsed in 0.004438877105712891 seconds
error: plugin_host has exited unexpectedly, plugin functionality won't be available until Sublime Text has been restarted

[niosus]

Ugh. This is confusing. This means that there is some other reason for this. I wonder why I cannot reproduce this. It seems like the others also don't have problems with this. Is there anything that can be different on your side to cause such a behavior?

[silva6]

It looks has nothing different on my side. The debug info cannot give any useful information.

[silva6]

I found this bug always occur when rename the source file that has open in tab. If the file is not open in tab, it will not cause plugin_host exit.

[silva6]

Could it be the view cache that cause this bug?

[niosus]

I can rename a file that is currently opened with no problems on my Linux machines... Hmmm... It seems that on your side by whatever reason, when you rename file, the view would still be with the old name.

Could you try to wrap the needed lines in tools.py in a try-catch block? Catching FileNotFoundError? Although, it seems there is no error in your case when you check if the path exists.

What we are missing is what exactly crashes the plugin host. It should not crash plugin_host even if there is an uncaught exception thrown in my plugin. It should just crash the plugin itself, but not the host.

[simia]

Happens to me as well.

[niosus]

Are you both on osx?

[simia]

Nah I am on a legit Linux :) Since I can reproduce I'll try to investigate this later.

[niosus]

Wow. I seem to be doing something wrong then... Hate those bugs that don't happen everywhere :)

[simia]

It seems that everything works fine when I use ANF for changing file name. Sublime native rename breaks it:(

[niosus]

What is ANF?

[simia]

https://packagecontrol.io/packages/AdvancedNewFile But as far as I can tell ANF closes old file and reopens new one so this is little different.

[niosus]

I don't use this extension. I am just renaming the file from side bar using sublime text own rename. Still works.

[simia]

Ok. Try this: Rename file. Press ctrl+s.

[niosus]

It works, but when I rename the file it is not changed. I think it is closed and opened again instantly.

[silva6]

I am on macOS. I don't use this extension too, just renaming from side bar. Renaming opened file will always reproduce this.

[niosus]

@silva6 @simia does it happen on any file? Or should the file be big? Does it work the same on test files in the repository under tests?

[silva6]

Yes, it happen on any file:(

[niosus]

What if you disable use_libclang?

[silva6]

WOW, it works...When disable use_libclang, there is no breaks now !

[niosus]

Ok, at least this makes sense to me. This is not a solution to the problem, but at least this guarantees there is something wrong internally in libclang (again...). This is going to be hard to find. Especially considering that I still fail to reproduce this :( Which version of clang are you using?

[silva6]

Apple LLVM version 8.0.0 (clang-800.0.42.1) Target: x86_64-apple-darwin16.3.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin

[silva6]

It's ok when renaming file in cmd line. Which is the same as ANF that mentioned before.

[niosus]

We must be doing something differently. I don't get it. Why is it failing for you, but not for me?

[silva6]

Today I test different sublime text versions. I install all version from 3103 to 3126. No one can pass the test:(

[silva6]

I add a lot logs to debug this. But don't know where is the last line that cause crash as there is no trace stack

[niosus]

Again, I'm pretty sure: whatever happens inside the plugin can at max crash the plugin, not the host. The thing that crashes the host is the communication with libclang. Probably, the problem spans from the filename being changed while (or before) the libclang is called. So somewhere in this range. That is my best bet for now.

[niosus]

I am experimenting with a solution to this problem. Please pull dev branch and see if this works for you.

[niosus]

Please reopen if it still fails.

[simia]

Still fails for me. I was tracing it for a while. Crashed seemed to occur somewhere in call to self.show_errors(view, self.tu.diagnostics) after calling self.tu.reparse() in lib_complete.py. At least that's what I could understand. But I am not very familiar with sublime host plugin interactions so this may be false lead.

[niosus]

Just to test, can you disable showing errors and try again?

[simia]

Yeah that seems to solve the problem. Didn't think of trying that :D

[niosus]

Ugh. Ok. Let me try something...

[niosus]

What about now? I am checking if the view is valid before showing errors now...

[simia]

Nope, still fails. Seems that sublime considers view valid.

[niosus]

Ooops, of course, I forgot to actually check if the path exists. Please try again.

[simia]

Hmm still crashes. The difference is that I don't get python trace of crash. Just dead plugin_host notification: error: plugin_host has exited unexpectedly, plugin functionality won't be available until Sublime Text has been restarted

I've tried moving you check to the begining of update() function. Still crashes. Maybe view object is somehow broken?

[simia]

Ok some more test: View seems to be valid and it point to the new file (after name change).

[niosus]

Ok, I'm clueless again... Can you add a line to output the name of the file before showing errors? Is it an old or a new one?

[niosus]

Too late with my question. :) What the heck? It is a valid view. What causes it to crash?

[simia]

My trace (print trace FTW!): x y y1 error: plugin_host has exited unexpectedly, plugin functionality won't be available until Sublime Text has been restarted

def errors_from_output(self, output):
        """Parse errors received from diagnostics of a translation unit.

        This is used with libclang.

        Args:
            output (diagnostics): diagnostics from a translation unit

        Returns:
            list(dict): a list of parsed errors
        """
        print("y")
        errors = []
        print("y1")
        for diag in output:
            print("y2")
            print(diag)

Looks like using DiagIterator causes problem.

[niosus]

Ok, cool. This is something. Then it's either line 344 or line 349 of cindex[num].py. Most probably 344. There is access to unrestricted C library to get length.

[simia]

I think it's line 2518: diag = conf.lib.clang_getDiagnostic(self.tu, key)
I am leaving for a weekend. We can pick this up on Monday.