Access to local socket interface for creation of plugins

[samdmarshall]

A number of utilities (code-completion, type-checks, etc) require access to local sockets to function. Typically they run a separate process that acts as a server and individual requests are made per file as things get changed. To facilitate more IDE-like functionality, an interface needs to be provided to allow for this socket access. This can be done via lua, but requires additional packages to be installed via luarocks. Unfortunately, the version of lua we are using to interface with Go-lang in the plugins does not support sockets or installation of additional packages from luarocks. To achieve this functionality, that interface will need to be provided by micro's Go-lang code instead. This would enable a whole new class of features via plugins and would be an overall net benefit to this application.

(We have talked about this on gitter chat but wanted to create an issue about it so that this could be tracked as that functionality gets added)

[GeigerCounter]

@samdmarshall I'm for this if you'll write the documentation for it. I don't understand sockets or Lua well enough to document this. And "the source is the documentation" is probably insufficient for this.

[zyedidia]

I've been thinking about overhauling the plugin system and in fact have been considering using Anko instead of Lua. Anko doesn't have the name recognition that lua has but it has a better interface to Go. It's a nice language based on Go (Lua has some strange syntax and other quirks) and it is very simple to expose Go's entire standard library to Anko, so plugins would be able to access the net package in the stdlib to run sockets for example. It can also access other Go packages, but they have to be precompiled into the micro binary.

This would be a pretty large change and if we switch to Anko, I think it would be good to keep the Lua support for backwards compatibility, at least until all the current plugins are converted, and possibly for ever.

I've experimented a bit with it and it's pretty easy to call functions from modules and whatnot so it wouldn't be too hard to implement. Let me know what you think. You can find code snippets for what anko's syntax looks like here.

[GeigerCounter]

@zyedidia Yes please. I need more curly brackets in my life. ( Also I'm helplessly bad at Lua. )

[samdmarshall]

@GeigerCounter i'm not sure what documentation you are asking me to write for this, for a baseline we need socket support to be able to take advantage of tools that would give micro a lot more IDE-esque functionality.

@zyedidia I am not sure about the change, I know very little about Lua and nothing about Anko. I can see the reasoning for wanting to switch to a scripting language that is more aligned with the editor's native language; however I think without more name recognition it will be harder to write plugins for it. Lua has name recognition and enough documentation in the wild that it is easy to start writing a plugin without knowing anything. Anko presents a higher difficulty curve for doing so, which will probably yield fewer third party plugins.

I think that the existing Lua plugin behavior keeps true to micro's mission statement of being simple and easy to deal with. The existing plugin API is extremely extensible as it is, and additional (major) features like this would require support from micro itself to do these things (eg: socket support). So I think keeping Lua would keep the number of dependencies in check as well as installation footprint. If more dependencies are needed then I think that the version of Lua we are using should be patched to allow packages from luarocks instead of managing them all inside of the editor.

[GeigerCounter]

@samdmarshall Wouldn't having a package manager cause the same or greater level of dependency hell?

@zyedidia I'm not sure about changing to Anko, but I know that Lua is less than desirable for making plugins in. The API is very good and is rather transparent and this is nice, but Lua itself is rather a weak scripting language imo. ( Good for quick and dirty hacks and on the fly configuration techniques, far less good for maintainable programs and integrating communications, and complex data manipulation. It's also confusing to debug. ) Could we maybe use Ruby instead then? I like the way the Lua interpreter directly builds into and interfaces with micro, but couldn't that be accomplished with another scripting language? Also, I've gotten a bit confused. What do you mean when you say it has no name recognition?

[samdmarshall]

Putting the idea out there that Javascript might be worth looking at for a plugin language.

[zyedidia]

I'm experimenting with giving Lua access to the go stdlib. If you check out the better-lua branch, plugins can access most go packages via an import command:

local runtime = import("runtime")
local fmt = import("fmt")

fmt.Println(runtime.GOOS)

This would fix this issue because lua plugins would be able to access net and use the functions from there.

[zyedidia]

This should be possible now on master by accessing Go's net library (local net = import("net")). Let me know if further development is needed.