JRFC 29 - Go Node

[jbenet]

I've been formulating ideas for a mad science experiment, and recently I've heard friends want to do this too, so let's do it.

Make Go more like Node

There are tons of subtleties that will go into this, but the basic idea is to re-imagine the Node programming system (#15, #27) with Go as a language (instead of javascript). Conversely, it can be seen as bringing sane package management and massive code reuse to Go -- a principle significantly undervalued in the mainstream Go ethos.[1]

Here is a shortlist of related features or principles I'd like to add to Go:

• sane package management (npm + ipfs, as in #27)
• ipfs pm instead of in-repo vendoring
• every file is a module
• sane versioning of code
• abandoning the package keyword
• require explicit, named imports (import foo "<path>")
• sane importing of multiple versions of the same module
• dealing with init (source of side-effect issues)
• reducing export surface area
• maximizing code reusability (minimizing "import friction")
• a node-like shell

Perhaps I could summarize this idea as:

Go with Modules, not Packages

to be continued

Note: this is just planting a flag. I will be growing this document over time.

---

[1] I should note that I (think I) understand why this is undervalued. It has more to do with how the Go team works (and how software engineering at Google works) than what makes a good programming system in the abstract. One of the guiding principles of Go is to focus on what works in practice, not on abstract ideals -- so it makes perfect sense. But, unlike software development inside Google, the open source world is a very messy place where code gets designed, written, maintained, abandoned, and forked by many different people. Package Management and Maximizing Code Reuse have become important staples of would-be open source hackers with big goals, little time, and the absence of a software engineer army.

[jbenet]

cc @dcposch -- @feross said you are thinking of this too

[shazow]

But, unlike software development inside Google, the open source world is a very messy place where code gets designed, written, maintained, abandoned, and forked by many different people.

You're underestimating how messy Google's internal code ecosystem is. :)

I'm sure everyone already knows about gopkg.in, but worth mentioning.

I find it's best to think of Go as a minimal possible collection of good and bad ideas that seem to work well together in practice. It's like a forcing function towards writing a very specific style of code, architected in very specific ways. It makes for very uniform designs across different types of engineers.

(That is not to say that the ideas here are not interesting, but I expect there will be a lot of resistance from the core community.)

[franleplant]

This seems very interesting!

[vijayee]

This needs to happen yesterday.

[jbenet]

IPFS -- http://ipfs.io -- has made tremendous progress so we may be able to try something like this later this year. reach out to me if you're interested. i foresee being able to devote time to this in mid-summer.

[kaustavha]

I also had this thought! Glad to know other people think it makes sense too.

Take a look at luvit.io and the luvit system. The point of it is to take lua which is a simple enough scripting language and the luajit is great for embedded systems or cases when you dont wanna take up too many resources (e.g. monitoring agents) and make it easy to write things in it. I had to learn it and basically went off of nodejs docs. So these guys have built a package manager like npm(lit), lib uv bindings for lua (luv), luvi(an inbetween project/application core?) and then theres a few other bindings for important things like ssl. Many of which may be trivially simple to do if Go comes with it, just wrap it in something and offer up a nodelike api. There's an irc channel called #luvit if you wanted to ask questions. https://github.com/luvit

[bcomnes]

hallelujah 👍

[jbenet]

@kaustavha wow luvit looks so exciting. i want to try it out soon

[burdges]

Has anyone written a good blog post about why Node does this correctly? I've never used Node myself.

There are several languages that might be more open than Go to experiments in doing module/package management better. Idris' developers have made comments to the effect that they do not know how to do it themselves. Rust's cargo sounds closer to this already and usually they're open to change. Go seems pretty fixed on doing things the way that fits well in a huge corporate ecosystem, like banning conditional compilation.

[bcomnes]

every file is a module

Is this different than require('name/any/sub/file')

abandoning the package keyword

Does this mean rename package.json to something else?

sane importing of multiple versions of the same module

You mean, importing the same module at different versions into another module?

[jbenet]

Is this different than require('name/any/sub/file')

no, precisely the same.

Does this mean rename package.json to something else?

no, the package keyword in Go.

You mean, importing the same module at different versions into another module?

yes.

---

since writing this, i believe a strict golang subset could be written to support most of this, armed with a strict preprocessor that fails compilation if any of the subset rules are violated.

[whyrusleeping]

I think there is one small change that could be made to go, that doesnt break anything, would be to implement deeper type matching.

If i have one package with a type MyType, and another package with the same type (fields have same types, or relative types), one package can use the types from another package.

This would allow interfaces to become even more flexible than they currently are, for example:

type Thing interface {
    Foo(Widget) error
    Bar() wobble.Badger
}

If in my package, i have a type like so:

type WorkThing struct {
   ...
}

func (wt *WorkThing) Foo(w zeeble.Widget) error {
    ...
}

func (wt *WorkThing) Bar()  wobblev2.Badger {
    ...
}

I would expect my type to implement Thing, as long as zeeble.Widget 'matches' Widget from the source package, and wobblev2.Badger matches wobble.Badger.

This would go a long ways to improving the versioning/vendoring story, and also help increase modularity immensely.