Closed entropitor closed 4 years ago
The handshake is documented here: https://github.com/hashicorp/go-plugin/blob/master/docs/internals.md#handshake
I think if I were working on go-plugin from scratch today I would go with this route. At the moment, I don’t see a large enough benefit to warrant the change. You note that it would be easier for other languages to write SDKs and I do agree with that, but I also think that outputting the one line isn’t too bad. If the documentation is incorrect, its easier to update that.
This is one of those situations where the end goal is certainly cleaner, but the backwards compatibility in between is very painful as we have to support the two handshake mechanisms for at least many years since we’ve already committed to plugin compatibility across many of the HashiCorp products. Given there would likely be many years of N+1 support (https://xkcd.com/927/) I’m inclined to just stick with the one admittedly weird approach for now.
Your suggestion is reasonable but I hope this helps share my viewpoint. Thanks for the suggestion.
@mitchellh Okay, I understand! I think the env variables for supported app versions should maybe also be listed on that page then as well
What is unclear to me about that wiki page is, should the handshake line be the first line on the stdout, the only line on stdout or can it be any line on stdout (and go-plugin will pick the first one that matches)? (I think it's especially here that it becomes a bit more cumbersome to write the SDK if it's "first line" as the SDK would have to block off the stdout from the plugin itself until the line is written and only then can stdout be used for real stuff)
(I was thinking an adapter application could do the translating between V1 and V2 and thus making it quite robust and backwards-compatible, outside of go-plugin, the applications or any of the SDKs) but given how e.g. terraform loads plugins, that might not be as easy to integrate. You would probably need a bash script as real extension then that loads the adapter application with an argument set to the real application you want to load and whether that application is V1 or V2)
These are good things to update the doc with but to answer your questions:
What is unclear to me about that wiki page is, should the handshake line be the first line on the stdout, the only line on stdout or can it be any line on stdout (and go-plugin will pick the first one that matches)?
First line, preferably only but I think any later lines are presently ignored.
The expectation is that the plugin process takes over stdout/stderr immediately on startup. That is what the Go plugin framework does, it expects you to call plugin.Serve
(or something like that) immediately on main
which overrides os.Stdout
so any loggers or whatever don't write to it.
(I was thinking an adapter application could do the translating between V1 and V2 and thus making it quite robust and backwards-compatible, outside of go-plugin, the applications or any of the SDKs)
Yeah its certainly possible no doubt. Its just one of those things that if the old way has to be maintained indefinitely, it feels a bit painful to introduce a second way. You went from one problem to two!
Okay I understand. Let's keep the current handshake!
Regarding the stdout/stderr, where should the output of plugins then go? Should it go to log files or just absorbed? I was thinking of potentially writing an SDK for another language and this is one of the last things that is unclear to me from reading the source code, what the SDK does exactly with stdout/stderr.
If it goes to stderr then the host system will log it, so that is the recommended place for logs! Otherwise yes a file. Or sometimes what we do is launch our plugins with extra file descriptors to get access to the original TTY too.
Instead of the complicated and undocumented handshake protocol, use a grpc server that is hosted by the application accepting plugins and pass just a single env variable to those applications with the address of that server.
Ideally the address is served over TCP for both windows support and because the go implementation over UDS is not following the spec https://github.com/grpc/grpc-go/issues/2628 at the moment (so plugins in rust / typescript / javascript might not work)
Process:
For backwards compatibility reasons, it's probably possible to write a "adapter" binary that starts the real plugin and adapts between v1 and this v2 (this proposal) of the handshake protocol. Given this adapter application can be written once in 1 language, it would be much easier to write SDKs for other languages without necessarily breaking the protocol.