snowkit / sublime_flow

Sublime Text support for Haxe flow build tool
5 stars 1 forks source link

Auto-completion broken and causing Sublime Text to hang #8

Closed anissen closed 7 years ago

anissen commented 8 years ago

Here's a real-time GIF showing the problem in action:

nov 19 2015 21 22

The status bar reads "no info/hxml for flow file, caching..." (printet from here).

The console outputs this:

Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 359, in on_query_completions
    res = callback.on_query_completions(v, prefix, locations)
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 115, in on_query_completions
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 132, in completion
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 93, in refresh_info
  File "./json/__init__.py", line 319, in loads
  File "./json/decoder.py", line 352, in decode
  File "./json/decoder.py", line 370, in raw_decode
ValueError: No JSON object could be decoded

The HXML-part of the JSON that is parsed in refresh_info in flow.py seems to be correct:

"hxml": "-main LuxeApp\n-D no-compilation\n-D mac\n-D arch-64\n-D desktop\n-D hxcpp_static_std\n-D absolute-path\n-D snow_module_input_sdl\n-D snow_module_audio_openal\n-D snow_module_windowing_sdl\n-D luxe_native\n-D parcel_thread_disabled\n-D snow_native\n-D luxe\n-D hxcpp\n-D snow\n-cp /Users/nissen/code/snowkit/luxe/\n-cp /usr/local/lib/haxelib/hxcpp/3,2,180/\n-cp /Users/nissen/code/snowkit/snow/\n-cp src/\n-resource version@version\n-resource build@build\n-resource phoenix/defaults/shaders/vert.glsl@default.vert.glsl\n-resource phoenix/defaults/shaders/frag.glsl@default.frag.glsl\n-resource phoenix/defaults/shaders/frag.textured.glsl@default.frag.textured.glsl\n-resource phoenix/defaults/shaders/frag.bitmapfont.glsl@default.frag.bitmapfont.glsl\n-resource luxe/defaults/fonts/default.png@default.png\n-resource luxe/defaults/fonts/default.fnt@default.fnt\n--macro snow.system.module.Module.set(\"Audio\",\"snow.core.native.audio.Audio\")\n--macro snow.system.module.Module.set(\"IO\",\"snow.core.native.io.IO\")\n--macro snow.system.module.Module.set(\"Input\",\"snow.core.native.input.Input\")\n--macro snow.system.module.Module.set(\"Windowing\",\"snow.core.native.window.Windowing\")\n--macro snow.system.module.Module.set(\"Assets\",\"snow.core.native.assets.Assets\")\n--macro snow.system.module.Module.set(\"Input\",\"snow.modules.sdl.Input\")\n--macro snow.system.module.Module.set(\"Windowing\",\"snow.modules.sdl.Windowing\")\n--macro snow.system.module.Module.set(\"Audio\",\"snow.modules.openal.Audio\")\n-cp bin/mac64.build/haxe\n-cpp bin/mac64.build/cpp\n",

... and the complete JSON-data is valid JSON.


I'm running OS X El Capitan with Haxe 3.2.1 and the newest flow, sublime_flow and sublime_haxe. The problem started occurring after I upgraded to El Capitan, but not immediately after.

hamaluik commented 8 years ago

I can confirm the exact same behaviour but on OSX Yosemite, with (to my knowledge) nothing changing on the system from before breaking to after breaking.

ruby0x1 commented 8 years ago

This is tricky to debug given that it's working fine for me on both my machines! I'm running sublime 3083 btw. Can either of you test re-installing the package? It almost looks as if the newer commit on 28 Oct may influence this but since I'm unable to get it to misbehave I can only start with narrowing down the obvious - a lot of which anissen has done on the validation front.

anissen commented 8 years ago

I'm running sublime 3083 btw.

Me too.

Can either of you test re-installing the package?

I've tried reinstalling sublime_flow but the issue persisted.

ruby0x1 commented 8 years ago

Can you update both this and sublime_haxe and try again?

anissen commented 8 years ago

Can you update both this and sublime_haxe and try again?

I've updated both plug-ins and the issue still remains :cry:

ruby0x1 commented 8 years ago

Check the console (sublime, view -> show console) for a message about how long completion took. I suspect this is some configuration issue that has nothing to do with the plugin itself.

anissen commented 8 years ago

Here's my console from start-up:

startup, version: 3083 osx x64 channel: stable
executable: /Applications/Sublime Text.app/Contents/MacOS/Sublime Text
working dir: /
packages path: /Users/nissen/Library/Application Support/Sublime Text 3/Packages
state path: /Users/nissen/Library/Application Support/Sublime Text 3/Local
zip path: /Applications/Sublime Text.app/Contents/MacOS/Packages
zip path: /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages
ignored_packages: ["EJS", "Jade", "NodeEval", "sublime-text-2-buildview", "SublimeLinter", "Vintage"]
pre session restore time: 0.336068
first paint time: 0.424884
startup time: 0.480585
reloading plugin Default.block
reloading plugin Default.comment
reloading plugin Default.copy_path
reloading plugin Default.delete_word
reloading plugin Default.detect_indentation
reloading plugin Default.duplicate_line
reloading plugin Default.echo
reloading plugin Default.exec
reloading plugin Default.fold
reloading plugin Default.font
reloading plugin Default.goto_line
reloading plugin Default.history_list
reloading plugin Default.indentation
reloading plugin Default.kill_ring
reloading plugin Default.mark
reloading plugin Default.new_templates
reloading plugin Default.open_context_url
reloading plugin Default.open_file_settings
reloading plugin Default.open_in_browser
reloading plugin Default.pane
reloading plugin Default.paragraph
reloading plugin Default.paste_from_history
reloading plugin Default.quick_panel
reloading plugin Default.save_on_focus_lost
reloading plugin Default.scroll
reloading plugin Default.set_unsaved_view_name
reloading plugin Default.side_bar
reloading plugin Default.sort
reloading plugin Default.swap_line
reloading plugin Default.switch_file
reloading plugin Default.symbol
reloading plugin Default.transform
reloading plugin Default.transpose
reloading plugin Default.trim_trailing_white_space
reloading plugin CSS.css_completions
reloading plugin Diff.diff
reloading plugin HTML.encode_html_entities
reloading plugin HTML.html_completions
reloading plugin 0_package_control_loader.00-package_control
reloading plugin 0_package_control_loader.01-pygments
reloading plugin 0_package_control_loader.02-bz2
reloading plugin 0_package_control_loader.50-markupsafe
reloading plugin 0_package_control_loader.50-python-markdown
reloading plugin 0_package_control_loader.51-python-jinja2
reloading plugin 0_package_control_loader.55-mdpopups
reloading plugin ANSIescape.ansi
reloading plugin Fix Mac Path.FixPath
reloading plugin GitGutter.git_gutter
reloading plugin GitGutter.git_gutter_change
reloading plugin GitGutter.git_gutter_compare
reloading plugin GitGutter.git_gutter_events
reloading plugin GitGutter.git_gutter_handler
reloading plugin GitGutter.git_helper
reloading plugin GitGutter.view_collection
reloading plugin Jekyll.jekyll
reloading plugin Markdown Preview.helper
reloading plugin Markdown Preview.markdown_settings
reloading plugin Markdown Preview.markdown_wrapper
reloading plugin Markdown Preview.MarkdownPreview
reloading plugin Package Control.1_reloader
reloading plugin Package Control.2_bootstrap
reloading plugin Package Control.Package Control
reloading plugin PlainNotes.jotter
reloading plugin PlainNotes.note_headings
reloading plugin PlainNotes.note_support
reloading plugin PlainNotes.note_todo
reloading plugin PlainNotes.notes
reloading plugin PlainNotes.notes_index
reloading plugin PlainTasks.PlainTasks
reloading plugin sublime_flow.flow
[flow] loaded show status
[flow] loaded set target build
[flow] loaded set project file
[flow] loaded run build
[flow] loaded generate import
<sublime_flow.flow.FlowProject object at 0x1029eeed0>
[flow] __init__
reloading plugin sublime_flow.haxe_parse_completion_list
reloading plugin sublime_haxe.haxe_completion
hello haxe completion
[haxe completion] __init__
reloading plugin TodoReview.TodoReview
plugins loaded
Package Control: Skipping automatic upgrade, last run at 2015-11-22 17:29:40, next run at 2015-11-22 18:29:40 or after

And setting a flow-file:

[flow] set flow file to /Users/nissen/code/snowkit/hexbash/project.flow
[flow] refresh info/hxml on /Users/nissen/code/snowkit/hexbash/project.flow
Traceback (most recent call last):
  File "./json/decoder.py", line 368, in raw_decode
  File "./json/scanner.py", line 63, in _scan_once
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 535, in run_
    return self.run()
  File "commands.flow_set_project_file in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 11, in run
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 63, in set_flow_file
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 94, in refresh_info
  File "./json/__init__.py", line 319, in loads
  File "./json/decoder.py", line 352, in decode
  File "./json/decoder.py", line 370, in raw_decode
ValueError: No JSON object could be decoded

And triggering auto-completion:

[flow] refresh info/hxml on /Users/nissen/code/snowkit/hexbash/project.flow
Traceback (most recent call last):
  File "./json/decoder.py", line 368, in raw_decode
  File "./json/scanner.py", line 63, in _scan_once
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 359, in on_query_completions
    res = callback.on_query_completions(v, prefix, locations)
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 116, in on_query_completions
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 133, in completion
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 94, in refresh_info
  File "./json/__init__.py", line 319, in loads
  File "./json/decoder.py", line 352, in decode
  File "./json/decoder.py", line 370, in raw_decode
ValueError: No JSON object could be decoded
[flow] refresh info/hxml on /Users/nissen/code/snowkit/hexbash/project.flow
Traceback (most recent call last):
  File "./json/decoder.py", line 368, in raw_decode
  File "./json/scanner.py", line 63, in _scan_once
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 359, in on_query_completions
    res = callback.on_query_completions(v, prefix, locations)
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 116, in on_query_completions
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 133, in completion
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 94, in refresh_info
  File "./json/__init__.py", line 319, in loads
  File "./json/decoder.py", line 352, in decode
  File "./json/decoder.py", line 370, in raw_decode
ValueError: No JSON object could be decoded
ruby0x1 commented 8 years ago

Can you uncomment and paste this line? https://github.com/snowkit/sublime_flow/blob/master/flow.py#L91

I know you pasted the HXML, but I want to see the full log please :)

hamaluik commented 8 years ago

Seems to be fixed for me, I'm getting things like the following:

[flow] saved file for completion
[haxe completion] complete
[flow] restore file post completion
[flow] completion took 0.642874002456665
ruby0x1 commented 8 years ago

I'm still finding and fixing some issues that I found that might be related but is hard to tell. The JSON failing to parse is a red flag though.

ruby0x1 commented 8 years ago

I've pushed the remaining fixes for now as well, let me know if that changes anything.

Quite rarely, I've managed to have it hang completely - but killall haxe (i.e kill the completion cache server) returns back to sublime, so the hanging is inside of the haxe executable. While it was hanging I sampled the app using the mac activity monitor, and it was in a stack inside neko regex stuff. I'm trying to track down a more reproducible case, testing against the latest haxe.

anissen commented 8 years ago

I've updated to the latest sublime_flow but it didn't fix my issue.

I've uncommented line 91 in flow.py but for some reason, I couldn't get sublime text to accept the changed package (some encoding error).

The error I'm getting seems to occur at line 89 (in decode) so I think I would need to write out the result of run_process(cmd) instead.

However, running haxelib run flow info web --project project.flow yields the following JSON:

{
  "hxml": "-main LuxeApp\n-D no-compilation\n-D web\n-D arch-web\n-D source-map-content\n-D hxcpp_static_std\n-D absolute-path\n-D snow_module_audio_howlerjs\n-D luxe_web\n-D snow_web\n-D luxe\n-D snow\n-cp /Users/nissen/code/snowkit/luxe/\n-cp /Users/nissen/code/snowkit/snow/\n-cp src/\n-resource version@version\n-resource build@build\n-resource phoenix/defaults/shaders/vert.glsl@default.vert.glsl\n-resource phoenix/defaults/shaders/frag.glsl@default.frag.glsl\n-resource phoenix/defaults/shaders/frag.textured.glsl@default.frag.textured.glsl\n-resource phoenix/defaults/shaders/frag.bitmapfont.glsl@default.frag.bitmapfont.glsl\n-resource luxe/defaults/fonts/default.png@default.png\n-resource luxe/defaults/fonts/default.fnt@default.fnt\n--macro snow.system.module.Module.set(\"Audio\",\"snow.core.web.audio.Audio\")\n--macro snow.system.module.Module.set(\"IO\",\"snow.core.web.io.IO\")\n--macro snow.system.module.Module.set(\"Input\",\"snow.core.web.input.Input\")\n--macro snow.system.module.Module.set(\"Windowing\",\"snow.core.web.window.Windowing\")\n--macro snow.system.module.Module.set(\"Assets\",\"snow.core.web.assets.Assets\")\n--macro snow.system.module.Module.set(\"Audio\",\"snow.modules.howlerjs.Audio\")\n-cp bin/web.build/haxe\n-js bin/web/hexbash.js\n",
  "paths": {
    "android": {
      "project": "project",
      "libabi": "armeabi"
    },
    "ios": {
      "project": "ios.project"
    },
    "output": "bin/web/",
    "build": "bin/web.build/",
    "files": "bin/web/",
    "binary": {
      "source": "LuxeApp",
      "path": "bin/web/",
      "name": "hexbash",
      "full": "bin/web/hexbash"
    }
  },
  "targets_known": [
    "mac",
    "windows",
    "linux",
    "ios",
    "android",
    "web"
  ],
  "targets_invalid": [
    "linux",
    "windows"
  ]
}
ruby0x1 commented 8 years ago

Oh yea, if you want to change / debug, just uninstall the plugin via package control, then clone it into Sublime -> Preferences -> Browse Packages. as long as it's in the packages folder as a cloned repo folder it will pick it up, then hitting save inside of flow.py it will reload the flow plugin each save (but that will clear the flow file, so you set it each time you test or hardcode the path in the init to iterate fast).

anissen commented 8 years ago

Thanks for the help -- I'm getting this output from flow.py:

[flow] json source: (B{
  "hxml": "-main LuxeApp\n-D no-compilation\n-D mac\n-D arch-64\n-D desktop\n-D hxcpp_static_std\n-D absolute-path\n-D snow_module_input_sdl\n-D snow_module_audio_openal\n-D snow_module_windowing_sdl\n-D luxe_native\n-D parcel_thread_disabled\n-D snow_native\n-D luxe\n-D gesluxe\n-D hxcpp\n-D snow\n-cp /Users/nissen/code/snowkit/luxe/\n-cp /Users/nissen/code/snowkit/Gesluxe/\n-cp /usr/local/lib/haxelib/hxcpp/3,2,180/\n-cp /Users/nissen/code/snowkit/snow/\n-cp src/\n-resource version@version\n-resource build@build\n-resource phoenix/defaults/shaders/vert.glsl@default.vert.glsl\n-resource phoenix/defaults/shaders/frag.glsl@default.frag.glsl\n-resource phoenix/defaults/shaders/frag.textured.glsl@default.frag.textured.glsl\n-resource phoenix/defaults/shaders/frag.bitmapfont.glsl@default.frag.bitmapfont.glsl\n-resource luxe/defaults/fonts/default.png@default.png\n-resource luxe/defaults/fonts/default.fnt@default.fnt\n--macro snow.system.module.Module.set(\"Audio\",\"snow.core.native.audio.Audio\")\n--macro snow.system.module.Module.set(\"IO\",\"snow.core.native.io.IO\")\n--macro snow.system.module.Module.set(\"Input\",\"snow.core.native.input.Input\")\n--macro snow.system.module.Module.set(\"Windowing\",\"snow.core.native.window.Windowing\")\n--macro snow.system.module.Module.set(\"Assets\",\"snow.core.native.assets.Assets\")\n--macro snow.system.module.Module.set(\"Input\",\"snow.modules.sdl.Input\")\n--macro snow.system.module.Module.set(\"Windowing\",\"snow.modules.sdl.Windowing\")\n--macro snow.system.module.Module.set(\"Audio\",\"snow.modules.openal.Audio\")\n-cp bin/mac64.build/haxe\n-cpp bin/mac64.build/cpp\n",
  "paths": {
    "android": {
      "project": "project",
      "libabi": "armeabi"
    },
    "ios": {
      "project": "ios.project"
    },
    "output": "bin/mac64/",
    "build": "bin/mac64.build/",
    "files": "bin/mac64/Hexmage.app/Contents/Resources/",
    "binary": {
      "source": "LuxeApp",
      "path": "bin/mac64/Hexmage.app/Contents/MacOS/",
      "name": "Hexmage",
      "full": "bin/mac64/Hexmage.app/Contents/MacOS/Hexmage"
    }
  },
  "targets_known": [
    "mac",
    "windows",
    "linux",
    "ios",
    "android",
    "web"
  ],
  "targets_invalid": [
    "linux",
    "windows"
  ]
}

I tried running this command in Python via the console and got the same result!

>>> json.loads('{  "hxml": "-main LuxeApp\n-D no-compilation\n-D web\n-D arch-web\n-D source-map-content\n-D hxcpp_static_std\n-D absolute-path\n-D snow_module_audio_howlerjs\n-D luxe_web\n-D snow_web\n-D luxe\n-D snow\n-cp /Users/nissen/code/snowkit/luxe/\n-cp /Users/nissen/code/snowkit/snow/\n-cp src/\n-resource version@version\n-resource build@build\n-resource phoenix/defaults/shaders/vert.glsl@default.vert.glsl\n-resource phoenix/defaults/shaders/frag.glsl@default.frag.glsl\n-resource phoenix/defaults/shaders/frag.textured.glsl@default.frag.textured.glsl\n-resource phoenix/defaults/shaders/frag.bitmapfont.glsl@default.frag.bitmapfont.glsl\n-resource luxe/defaults/fonts/default.png@default.png\n-resource luxe/defaults/fonts/default.fnt@default.fnt\n--macro snow.system.module.Module.set(\"Audio\",\"snow.core.web.audio.Audio\")\n--macro snow.system.module.Module.set(\"IO\",\"snow.core.web.io.IO\")\n--macro snow.system.module.Module.set(\"Input\",\"snow.core.web.input.Input\")\n--macro snow.system.module.Module.set(\"Windowing\",\"snow.core.web.window.Windowing\")\n--macro snow.system.module.Module.set(\"Assets\",\"snow.core.web.assets.Assets\")\n--macro snow.system.module.Module.set(\"Audio\",\"snow.modules.howlerjs.Audio\")\n-cp bin/web.build/haxe\n-js bin/web/hexbash.js\n",  "paths": {    "android": {      "project": "project",      "libabi": "armeabi"    },    "ios": {      "project": "ios.project"    },    "output": "bin/web/",    "build": "bin/web.build/",    "files": "bin/web/",    "binary": {      "source": "LuxeApp",      "path": "bin/web/",      "name": "hexbash",      "full": "bin/web/hexbash"    }  },  "targets_known": [    "mac",    "windows",    "linux",    "ios",    "android",    "web"  ],  "targets_invalid": [    "linux",    "windows"  ]}')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Invalid control character at: line 1 column 26 (char 25)
ruby0x1 commented 8 years ago

Yea that looks like a BOM character and a color sequence, interesting. So somewhere that invalid sequence up front is sneaking into the string, do you use any particular shell other than the default?

anissen commented 8 years ago

Yea that looks like a BOM character and a color sequence, interesting.

I think that may be the console in Sublime Text that inserted those characters? Anyway, on my laptop and work computer I've tried running the Python code to load just the HXML-part of the JSON (as shown above). On both computers (both running Python 2.7) I'm getting the error shown above. My Sublime Text runs Python 3.3.3

Doing some further tests, it seems that Python throws up if trying to parse JSON that contains newlines or \". Here's some command line tests showing the issue:

>>> json.loads('{ "hxml": "blah" }')
{u'hxml': u'blah'}

This works (do'h).

>>> json.loads('{ "hxml": "blah\n" }')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Invalid control character at: line 1 column 16 (char 15)

Error parsing \n (cannot be escaped as \\n)

>>> json.loads('{ "hxml": "bl\"ah" }')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 1 column 15 (char 14)
>>> json.loads('{ "hxml": "bl\"a\"h" }')

Error parsing \" (can be escaped as \\").

Later today, I can try changing flow.py to replace \n with space and \" with \\" and see if that works.

Edit: Haxe seems to have similar problems parsing the HXML: http://try.haxe.org/#79Fe2

ruby0x1 commented 8 years ago

Yea, I find it curious however that for the last say year+ this hasn't come up before (and still isn't for 99%).

See the latest flow and sublime_flow commits, let me know if you see anything weird.

ImDeity commented 8 years ago

I've also been using the sublime_flow plugin in ST for months without issue but even after the latest updates to flow and the ST plugins, I'm still not getting any auto-completion:

[flow] refresh info/hxml on C:\Users\Deity\Documents\AeroFS\Documents\Dev\haxe\imhex\project.flow
Traceback (most recent call last):
  File "./json/decoder.py", line 368, in raw_decode
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Sublime Text 3\sublime_plugin.py", line 359, in on_query_completions
    res = callback.on_query_completions(v, prefix, locations)
  File "flow in C:\Users\Deity\AppData\Roaming\Sublime Text 3\Installed Packages\sublime_flow.sublime-package", line 121, in on_query_completions
  File "flow in C:\Users\Deity\AppData\Roaming\Sublime Text 3\Installed Packages\sublime_flow.sublime-package", line 142, in completion
  File "flow in C:\Users\Deity\AppData\Roaming\Sublime Text 3\Installed Packages\sublime_flow.sublime-package", line 94, in refresh_info
  File "./json/__init__.py", line 319, in loads
  File "./json/decoder.py", line 352, in decode
  File "./json/decoder.py", line 370, in raw_decode
ValueError: No JSON object could be decoded
SublimeLinter: annotations activated: <builtin> 

This is on Windows 8 with Haxe 3.2.0 and ST build 3083 using Flow commit b8520ea244d62c3c929439eb7709451c01b1c5b4.

ruby0x1 commented 8 years ago

For reference, the initial step (set as active flow file) is most relevant, everything is hinged on the set step working initially so any errors that result from trying to complete against invalid state may be misleading.

@ImDeity @anissen I pushed the logging to the console of the received json so it's easier to debug. I suspect whatever made the py JSON parser sporadically pedantic on some machines is potentially disliking the windows paths in the string.

@ImDeity can you paste the block that the sublime consoles prints out between these;

[flow] json source: >>
<<
ImDeity commented 8 years ago

Updated to latest sublime_flow and on setting the active flow file, this is in console:

[flow] set flow file to C:\Users\Deity\Documents\AeroFS\Documents\Dev\haxe\imhex\project.flow
[flow] refresh info/hxml on C:\Users\Deity\Documents\AeroFS\Documents\Dev\haxe\imhex\project.flow
[flow] json source: >>
C:\HaxeToolkit\haxe\lib\flow\git\src\flow\cmd\info\info.js:43

        .replace(/[\"]/g, '\\\"')

        ^

TypeError: Cannot read property 'replace' of undefined

    at Object.internal.escape_json (C:\HaxeToolkit\haxe\lib\flow\git\src\flow\cmd\info\info.js:43:9)

    at Object.run (C:\HaxeToolkit\haxe\lib\flow\git\src\flow\cmd\info\info.js:18:25)

    at C:\HaxeToolkit\haxe\lib\flow\git\src\flow\flow.js:32:21

    at Object.verify (C:\HaxeToolkit\haxe\lib\flow\git\src\flow\cmd\info\info.js:32:5)

    at Object.flow.execute (C:\HaxeToolkit\haxe\lib\flow\git\src\flow\flow.js:29:13)

    at internal.run (C:\HaxeToolkit\haxe\lib\flow\git\src\flow\flow.js:92:18)

    at C:\HaxeToolkit\haxe\lib\flow\git\src\flow\util\haxelib.js:78:21

    at ChildProcess.<anonymous> (C:\HaxeToolkit\haxe\lib\flow\git\src\flow\util\process.js:56:26)

    at ChildProcess.emit (events.js:110:17)

    at maybeClose (child_process.js:1015:16)

<<
Traceback (most recent call last):
  File "./json/decoder.py", line 368, in raw_decode
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Program Files\Sublime Text 3\sublime_plugin.py", line 535, in run_
    return self.run()
  File "commands.flow_set_project_file in C:\Users\Deity\AppData\Roaming\Sublime Text 3\Installed Packages\sublime_flow.sublime-package", line 11, in run
  File "flow in C:\Users\Deity\AppData\Roaming\Sublime Text 3\Installed Packages\sublime_flow.sublime-package", line 63, in set_flow_file
  File "flow in C:\Users\Deity\AppData\Roaming\Sublime Text 3\Installed Packages\sublime_flow.sublime-package", line 94, in refresh_info
  File "./json/__init__.py", line 319, in loads
  File "./json/decoder.py", line 352, in decode
  File "./json/decoder.py", line 370, in raw_decode
ValueError: No JSON object could be decoded
ruby0x1 commented 8 years ago

@ImDeity before you update can you run flow info on the command line and see what happens? I'm basically wondering how your hxml can be invalid, that's not a state that's expected in flow.

After that, can you update flow, and then paste the same section (which presumably may have an invalid hxml node rather than a crash log)

ImDeity commented 8 years ago

Output from flow info within project folder: http://pastie.org/pastes/10576161/text?key=uyjpqetlby1knbuten8m2g

After: http://pastie.org/pastes/10576168/text?key=vvmhvrweiam9l305bomnaq

Crash log on neither one (they seem identical).

Output from setting active flow file after upgrade:

[flow] set flow file to C:\Users\Deity\Documents\AeroFS\Documents\Dev\haxe\imhex\project.flow
[flow] refresh info/hxml on C:\Users\Deity\Documents\AeroFS\Documents\Dev\haxe\imhex\project.flow
[flow] json source: >>
{
  "paths": {},
  "targets_known": [
    "mac",
    "windows",
    "linux",
    "ios",
    "android",
    "web"
  ],
  "targets_invalid": [
    "linux",
    "mac",
    "ios"
  ]
}

<<
Traceback (most recent call last):
  File "C:\Program Files\Sublime Text 3\sublime_plugin.py", line 535, in run_
    return self.run()
  File "commands.flow_set_project_file in C:\Users\Deity\AppData\Roaming\Sublime Text 3\Installed Packages\sublime_flow.sublime-package", line 11, in run
  File "flow in C:\Users\Deity\AppData\Roaming\Sublime Text 3\Installed Packages\sublime_flow.sublime-package", line 63, in set_flow_file
  File "flow in C:\Users\Deity\AppData\Roaming\Sublime Text 3\Installed Packages\sublime_flow.sublime-package", line 98, in refresh_info
KeyError: 'hxml'
ruby0x1 commented 8 years ago

That's definitely not right, but I consider that unrelated to this issue at hand. Can you open another issue, But start with pasting what sublimetext sees (since console seems fine) after commenting this line in flow itself (there is no rebuild step for flow or anything, just a quick test). https://github.com/underscorediscovery/flow/blob/master/src/flow/flow.js#L173

That should print more info (which will invalidate the sublime part, expectedly) but I'm curious why the state of your project when queried from sublime is somehow blank. I don't know a code path that can get there.

anissen commented 8 years ago

I really appreciate all your work in getting this to work. Unfortunately, it still doesn't :disappointed:

Here's my Sublime Text console output from flow:

(B{
  "hxml": "-main LuxeApp\\n-D no-compilation\\n-D mac\\n-D arch-64\\n-D desktop\\n-D hxcpp_static_std\\n-D absolute-path\\n-D snow_module_input_sdl\\n-D snow_module_audio_openal\\n-D snow_module_windowing_sdl\\n-D luxe_native\\n-D parcel_thread_disabled\\n-D snow_native\\n-D luxe\\n-D hxcpp\\n-D snow\\n-cp /Users/nissen/code/snowkit/luxe/\\n-cp /usr/local/lib/haxelib/hxcpp/3,2,180/\\n-cp /Users/nissen/code/snowkit/snow/\\n-cp src/\\n-resource version@version\\n-resource build@build\\n-resource phoenix/defaults/shaders/vert.glsl@default.vert.glsl\\n-resource phoenix/defaults/shaders/frag.glsl@default.frag.glsl\\n-resource phoenix/defaults/shaders/frag.textured.glsl@default.frag.textured.glsl\\n-resource phoenix/defaults/shaders/frag.bitmapfont.glsl@default.frag.bitmapfont.glsl\\n-resource luxe/defaults/fonts/default.png@default.png\\n-resource luxe/defaults/fonts/default.fnt@default.fnt\\n--macro snow.system.module.Module.set(\\\"Audio\\\",\\\"snow.core.native.audio.Audio\\\")\\n--macro snow.system.module.Module.set(\\\"IO\\\",\\\"snow.core.native.io.IO\\\")\\n--macro snow.system.module.Module.set(\\\"Input\\\",\\\"snow.core.native.input.Input\\\")\\n--macro snow.system.module.Module.set(\\\"Windowing\\\",\\\"snow.core.native.window.Windowing\\\")\\n--macro snow.system.module.Module.set(\\\"Assets\\\",\\\"snow.core.native.assets.Assets\\\")\\n--macro snow.system.module.Module.set(\\\"Input\\\",\\\"snow.modules.sdl.Input\\\")\\n--macro snow.system.module.Module.set(\\\"Windowing\\\",\\\"snow.modules.sdl.Windowing\\\")\\n--macro snow.system.module.Module.set(\\\"Audio\\\",\\\"snow.modules.openal.Audio\\\")\\n-cp bin/mac64.build/haxe\\n-cpp bin/mac64.build/cpp\\n",
  "paths": {
    "android": {
      "project": "project",
      "libabi": "armeabi"
    },
    "ios": {
      "project": "ios.project"
    },
    "output": "bin/mac64/",
    "build": "bin/mac64.build/",
    "files": "bin/mac64/hexbash.app/Contents/Resources/",
    "binary": {
      "source": "LuxeApp",
      "path": "bin/mac64/hexbash.app/Contents/MacOS/",
      "name": "hexbash",
      "full": "bin/mac64/hexbash.app/Contents/MacOS/hexbash"
    }
  },
  "targets_known": [
    "mac",
    "windows",
    "linux",
    "ios",
    "android",
    "web"
  ],
  "targets_invalid": [
    "linux",
    "windows"
  ]
}

And here's the error:

Traceback (most recent call last):
  File "./json/decoder.py", line 368, in raw_decode
  File "./json/scanner.py", line 63, in _scan_once
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 535, in run_
    return self.run()
  File "commands.flow_set_project_file in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 11, in run
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 63, in set_flow_file
  File "flow in /Users/nissen/Library/Application Support/Sublime Text 3/Installed Packages/sublime_flow.sublime-package", line 94, in refresh_info
  File "./json/__init__.py", line 319, in loads
  File "./json/decoder.py", line 352, in decode
  File "./json/decoder.py", line 370, in raw_decode
ValueError: No JSON object could be decoded

I figured that there was still newlines in the output from flow, so I changed flow/cmd/info/info.js to escape the entire JSON. Still throws an error though:

[flow] set flow file to /Users/nissen/code/snowkit/hexbash/project.flow
[flow] refresh info/hxml on /Users/nissen/code/snowkit/hexbash/project.flow
[flow] json source: >>
(B{\"hxml\":\"-main LuxeApp -D no-compilation -D mac -D arch-64 -D desktop -D hxcpp_static_std -D absolute-path -D snow_module_input_sdl -D snow_module_audio_openal -D snow_module_windowing_sdl -D luxe_native -D parcel_thread_disabled -D snow_native -D luxe -D hxcpp -D snow -cp /Users/nissen/code/snowkit/luxe/ -cp /usr/local/lib/haxelib/hxcpp/3,2,180/ -cp /Users/nissen/code/snowkit/snow/ -cp src/ -resource version@version -resource build@build -resource phoenix/defaults/shaders/vert.glsl@default.vert.glsl -resource phoenix/defaults/shaders/frag.glsl@default.frag.glsl -resource phoenix/defaults/shaders/frag.textured.glsl@default.frag.textured.glsl -resource phoenix/defaults/shaders/frag.bitmapfont.glsl@default.frag.bitmapfont.glsl -resource luxe/defaults/fonts/default.png@default.png -resource luxe/defaults/fonts/default.fnt@default.fnt --macro snow.system.module.Module.set(\\"Audio\\",\\"snow.core.native.audio.Audio\\") --macro snow.system.module.Module.set(\\"IO\\",\\"snow.core.native.io.IO\\") --macro snow.system.module.Module.set(\\"Input\\",\\"snow.core.native.input.Input\\") --macro snow.system.module.Module.set(\\"Windowing\\",\\"snow.core.native.window.Windowing\\") --macro snow.system.module.Module.set(\\"Assets\\",\\"snow.core.native.assets.Assets\\") --macro snow.system.module.Module.set(\\"Input\\",\\"snow.modules.sdl.Input\\") --macro snow.system.module.Module.set(\\"Windowing\\",\\"snow.modules.sdl.Windowing\\") --macro snow.system.module.Module.set(\\"Audio\\",\\"snow.modules.openal.Audio\\") -cp bin/mac64.build/haxe -cpp bin/mac64.build/cpp \",\"paths\":{\"android\":{\"project\":\"project\",\"libabi\":\"armeabi\"},\"ios\":{\"project\":\"ios.project\"},\"output\":\"bin/mac64/\",\"build\":\"bin/mac64.build/\",\"files\":\"bin/mac64/hexbash.app/Contents/Resources/\",\"binary\":{\"source\":\"LuxeApp\",\"path\":\"bin/mac64/hexbash.app/Contents/MacOS/\",\"name\":\"hexbash\",\"full\":\"bin/mac64/hexbash.app/Contents/MacOS/hexbash\"}},\"targets_known\":[\"mac\",\"windows\",\"linux\",\"ios\",\"android\",\"web\"],\"targets_invalid\":[\"linux\",\"windows\"]}

<<
Traceback (most recent call last):
  File "./json/decoder.py", line 368, in raw_decode
  File "./json/scanner.py", line 63, in _scan_once
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Applications/Sublime Text.app/Contents/MacOS/sublime_plugin.py", line 535, in run_
    return self.run()
  File "/Users/nissen/Library/Application Support/Sublime Text 3/Packages/sublime_flow/commands/flow_set_project_file.py", line 11, in run
    _flow_.set_flow_file( view.file_name() )
  File "/Users/nissen/Library/Application Support/Sublime Text 3/Packages/sublime_flow/flow.py", line 63, in set_flow_file
    self.refresh_info()
  File "/Users/nissen/Library/Application Support/Sublime Text 3/Packages/sublime_flow/flow.py", line 94, in refresh_info
    self.info_json = json.loads(self.info_json_src)
  File "./json/__init__.py", line 319, in loads
  File "./json/decoder.py", line 352, in decode
  File "./json/decoder.py", line 370, in raw_decode
ValueError: No JSON object could be decoded

Copy-paste the JSON directly into the Sublime Text (python) console gets this output:

>>> json.loads('{\"hxml\":\"-main LuxeApp -D no-compilation -D mac -D arch-64 -D desktop -D hxcpp_static_std -D absolute-path -D snow_module_input_sdl -D snow_module_audio_openal -D snow_module_windowing_sdl -D luxe_native -D parcel_thread_disabled -D snow_native -D luxe -D hxcpp -D snow -cp /Users/nissen/code/snowkit/luxe/ -cp /usr/local/lib/haxelib/hxcpp/3,2,180/ -cp /Users/nissen/code/snowkit/snow/ -cp src/ -resource version@version -resource build@build -resource phoenix/defaults/shaders/vert.glsl@default.vert.glsl -resource phoenix/defaults/shaders/frag.glsl@default.frag.glsl -resource phoenix/defaults/shaders/frag.textured.glsl@default.frag.textured.glsl -resource phoenix/defaults/shaders/frag.bitmapfont.glsl@default.frag.bitmapfont.glsl -resource luxe/defaults/fonts/default.png@default.png -resource luxe/defaults/fonts/default.fnt@default.fnt --macro snow.system.module.Module.set(\\"Audio\\",\\"snow.core.native.audio.Audio\\") --macro snow.system.module.Module.set(\\"IO\\",\\"snow.core.native.io.IO\\") --macro snow.system.module.Module.set(\\"Input\\",\\"snow.core.native.input.Input\\") --macro snow.system.module.Module.set(\\"Windowing\\",\\"snow.core.native.window.Windowing\\") --macro snow.system.module.Module.set(\\"Assets\\",\\"snow.core.native.assets.Assets\\") --macro snow.system.module.Module.set(\\"Input\\",\\"snow.modules.sdl.Input\\") --macro snow.system.module.Module.set(\\"Windowing\\",\\"snow.modules.sdl.Windowing\\") --macro snow.system.module.Module.set(\\"Audio\\",\\"snow.modules.openal.Audio\\") -cp bin/mac64.build/haxe -cpp bin/mac64.build/cpp \",\"paths\":{\"android\":{\"project\":\"project\",\"libabi\":\"armeabi\"},\"ios\":{\"project\":\"ios.project\"},\"output\":\"bin/mac64/\",\"build\":\"bin/mac64.build/\",\"files\":\"bin/mac64/hexbash.app/Contents/Resources/\",\"binary\":{\"source\":\"LuxeApp\",\"path\":\"bin/mac64/hexbash.app/Contents/MacOS/\",\"name\":\"hexbash\",\"full\":\"bin/mac64/hexbash.app/Contents/MacOS/hexbash\"}},\"targets_known\":[\"mac\",\"windows\",\"linux\",\"ios\",\"android\",\"web\"],\"targets_invalid\":[\"linux\",\"windows\"]}')
{'hxml': '-main LuxeApp -D no-compilation -D mac -D arch-64 -D desktop -D hxcpp_static_std -D absolute-path -D snow_module_input_sdl -D snow_module_audio_openal -D snow_module_windowing_sdl -D luxe_native -D parcel_thread_disabled -D snow_native -D luxe -D hxcpp -D snow -cp /Users/nissen/code/snowkit/luxe/ -cp /usr/local/lib/haxelib/hxcpp/3,2,180/ -cp /Users/nissen/code/snowkit/snow/ -cp src/ -resource version@version -resource build@build -resource phoenix/defaults/shaders/vert.glsl@default.vert.glsl -resource phoenix/defaults/shaders/frag.glsl@default.frag.glsl -resource phoenix/defaults/shaders/frag.textured.glsl@default.frag.textured.glsl -resource phoenix/defaults/shaders/frag.bitmapfont.glsl@default.frag.bitmapfont.glsl -resource luxe/defaults/fonts/default.png@default.png -resource luxe/defaults/fonts/default.fnt@default.fnt --macro snow.system.module.Module.set("Audio","snow.core.native.audio.Audio") --macro snow.system.module.Module.set("IO","snow.core.native.io.IO") --macro snow.system.module.Module.set("Input","snow.core.native.input.Input") --macro snow.system.module.Module.set("Windowing","snow.core.native.window.Windowing") --macro snow.system.module.Module.set("Assets","snow.core.native.assets.Assets") --macro snow.system.module.Module.set("Input","snow.modules.sdl.Input") --macro snow.system.module.Module.set("Windowing","snow.modules.sdl.Windowing") --macro snow.system.module.Module.set("Audio","snow.modules.openal.Audio") -cp bin/mac64.build/haxe -cpp bin/mac64.build/cpp ', 'paths': {'binary': {'full': 'bin/mac64/hexbash.app/Contents/MacOS/hexbash', 'path': 'bin/mac64/hexbash.app/Contents/MacOS/', 'name': 'hexbash', 'source': 'LuxeApp'}, 'ios': {'project': 'ios.project'}, 'build': 'bin/mac64.build/', 'android': {'project': 'project', 'libabi': 'armeabi'}, 'output': 'bin/mac64/', 'files': 'bin/mac64/hexbash.app/Contents/Resources/'}, 'targets_invalid': ['linux', 'windows'], 'targets_known': ['mac', 'windows', 'linux', 'ios', 'android', 'web']}

... where it works :angry:

Edit: But if I include the escape characters at the beginning I get the following!

>>> json.loads('(B{\"hxml\":\"-main LuxeApp -D no-compilation -D mac -D arch-64 -D desktop -D hxcpp_static_std -D absolute-path -D snow_module_input_sdl -D snow_module_audio_openal -D snow_module_windowing_sdl -D luxe_native -D parcel_thread_disabled -D snow_native -D luxe -D hxcpp -D snow -cp /Users/nissen/code/snowkit/luxe/ -cp /usr/local/lib/haxelib/hxcpp/3,2,180/ -cp /Users/nissen/code/snowkit/snow/ -cp src/ -resource version@version -resource build@build -resource phoenix/defaults/shaders/vert.glsl@default.vert.glsl -resource phoenix/defaults/shaders/frag.glsl@default.frag.glsl -resource phoenix/defaults/shaders/frag.textured.glsl@default.frag.textured.glsl -resource phoenix/defaults/shaders/frag.bitmapfont.glsl@default.frag.bitmapfont.glsl -resource luxe/defaults/fonts/default.png@default.png -resource luxe/defaults/fonts/default.fnt@default.fnt --macro snow.system.module.Module.set(\\"Audio\\",\\"snow.core.native.audio.Audio\\") --macro snow.system.module.Module.set(\\"IO\\",\\"snow.core.native.io.IO\\") --macro snow.system.module.Module.set(\\"Input\\",\\"snow.core.native.input.Input\\") --macro snow.system.module.Module.set(\\"Windowing\\",\\"snow.core.native.window.Windowing\\") --macro snow.system.module.Module.set(\\"Assets\\",\\"snow.core.native.assets.Assets\\") --macro snow.system.module.Module.set(\\"Input\\",\\"snow.modules.sdl.Input\\") --macro snow.system.module.Module.set(\\"Windowing\\",\\"snow.modules.sdl.Windowing\\") --macro snow.system.module.Module.set(\\"Audio\\",\\"snow.modules.openal.Audio\\") -cp bin/mac64.build/haxe -cpp bin/mac64.build/cpp \",\"paths\":{\"android\":{\"project\":\"project\",\"libabi\":\"armeabi\"},\"ios\":{\"project\":\"ios.project\"},\"output\":\"bin/mac64/\",\"build\":\"bin/mac64.build/\",\"files\":\"bin/mac64/hexbash.app/Contents/Resources/\",\"binary\":{\"source\":\"LuxeApp\",\"path\":\"bin/mac64/hexbash.app/Contents/MacOS/\",\"name\":\"hexbash\",\"full\":\"bin/mac64/hexbash.app/Contents/MacOS/hexbash\"}},\"targets_known\":[\"mac\",\"windows\",\"linux\",\"ios\",\"android\",\"web\"],\"targets_invalid\":[\"linux\",\"windows\"]}')
Traceback (most recent call last):
  File "./json/decoder.py", line 368, in raw_decode
  File "./json/scanner.py", line 63, in _scan_once
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "./json/__init__.py", line 319, in loads
  File "./json/decoder.py", line 352, in decode
  File "./json/decoder.py", line 370, in raw_decode
ValueError: No JSON object could be decoded
ruby0x1 commented 8 years ago

Yea you can't escape the whole JSON, that won't work. "hxml" would become invalid json as \"hxml\", the keys are strings.

�(B�[m this is still printed in the output which bothers me.

Are you on the latest commits? I removed the logging but since you had a clone you edit it back in and try, I'm fairly certain that the logic is sound now, I put the new lines and >> so that we could pinpoint if those characters come from the result, or if they come from sublime. The above means that somehow your actual result from flow has some preceding characters that are breaking the json. I suppose one thing to try is to strip() the source, see if it takes those characters out before moving on.

anissen commented 8 years ago

I've found the best, most elegant solution to my problem!

Changing line 94 to self.info_json = json.loads(self.info_json_src[6:]) :P

It does in fact work! So the problem was the wild escape characters. However, auto-completion is not looking too hot either: screen shot 2015-11-23 at 20 51 30 I suppose it might be related. But I have no idea why or how there's escape characters everywhere...

ruby0x1 commented 8 years ago

Yea, here's something you can run in the sublime console for debugging there,

import subprocess; t=subprocess.Popen(['haxe','-version'],stdout=subprocess.PIPE, stderr=subprocess.STDOUT).communicate()[0]; print(t.decode('utf-8'))

This just runs haxe -version but since that might not be found you could try /usr/local/bin/haxe instead, or any command. I'm curious if it does it from here too.

You might be able to mess with the decode option to see if you can output raw binary or hex to try and understand what byte sequence it's getting (instead of the replacement character).

anissen commented 8 years ago

Yea, here's something you can run in the sublime console for debugging there,

No escape chars:

>>> import subprocess; t=subprocess.Popen(['haxe','-version'],stdout=subprocess.PIPE, stderr=subprocess.STDOUT).communicate()[0]; print(t.decode('utf-8'))
3.2.1

You might be able to mess with the decode option to see if you can output raw binary or hex to try and understand what byte sequence it's getting (instead of the replacement character).

No decoding gives b'{.... I've tried a few others; utf-8-sig, utf-16, latin-1 that didn't give different results. I haven't been able to find a hex or binary encoding option.

ruby0x1 commented 8 years ago

Btw I meant encode rather, i.e convert the raw output to viewable bytes, so we can see what the stream looks like. Often you can google a byte sequence like BOM, color codes etc.

Here's what happens in this set of commands:

That's the end of the chain. I starred two things, above. First, maybe a custom build of haxelib or something is causing this, but it's somewhere to add logging and check if it's sane. Just run compile.hxml to rebuild that step. You can maybe pinpoint where exactly it gets transformed this way.

Second, the muted line, flow has verbose logging, forcing that to 3 or 4 instead of 0 might yield more information, you could look for the entry point and simple console.log('test') and try pinpoint where it's being mangled. This will generate invalid json (via prefixed logging in the output)