lib it out!

[colhom]

Main obstacle here: Most applications making use of this library will want to deal with go objects of some sort at runtime, rather than string literals which need to be unmarshalled, modified and re-marshalled in most use cases.

There are some architectural "additions" I'll venture a recommendations towards:

• An intermediate representation (IR) for a generated network layout that can then render to to any of our various output formats (table, terraform, cloudformation). This IR will be the "stable" interface this package will present to dependent codebases going forward.

Later on, perhaps we can extract the individual output formats into implementations behind a common interface and attempt to provide an abstraction layer for actually constructing these configs across providers. Given that there are a lot of different ways to slice that cake, I think we're better off (for now) settling on an expressive enough IR and leaving it up to those implementing on top of the library to construct their own output configurations from the generated network layout IR.

\cc @tschuy @brianredbeard

[tschuy]

libbing it out was definitely on the list (and the entire codebase should have every variable name replaced with U+1F4A9 -- this is definitely a prelim implementation). Those all sound like pretty sound architecture ideas, so I'll probably go down that route.

[tschuy]

@colhom check out the tip of the repo. it's now nicely split apart into subpackages, including

• an HTTP server
• outputs:
  • output/table
  • output/terraform
• subnet

the IR is in subnet/subnet.go, but it needs to be renamed (the structs don't really accurately represent what they really hold) -- #3. Adding a new output format is just throwing it into output/ as a subpackage, and then adding it to the format function map in the HTTP server/cli.

[razic]

/cc @colhom @tschuy

Main obstacle here: Most applications making use of this library will want to deal with go objects of some sort at runtime, rather than string literals which need to be unmarshalled, modified and re-marshalled in most use cases.

What do you think about implementing gRPC? The server no longer becomes "just" an HTTP server, but rather a gRPC server, which can be set up to listen over TCP, or UNIX domain sockets. gRPC would use protobufs which would provide a significant improvement in performance over typical payloads (JSON, XML, etc...). The gRPC model would enable beautiful interprocess communication. Other developers could integrate the code into their applications (with support for many different programming languages), using remote procedure calls that seem native.

Using gRPC, we could define the external API using the proto file syntax. Furthermore, gRPC has a plugin model. One super useful plugin for this case could be the https://github.com/grpc-ecosystem/grpc-gateway. This plugin provides a CLI command to start a HTTP reverse proxy that maps HTTP calls to RPC calls, from a given config file, in the scenario you don't want to use gRPC directly for IPC.

One really great example of this style of codebase is https://github.com/containerd/containerd. The ./api folder defines the gRPC API, which can be versioned. The API is then implemented by registering handlers when you start the gRPC server: https://github.com/containerd/containerd/blob/master/server/server.go#L111-L117

Also, wondering what the philosophy around writing tests is here, as there appears to be none of them ;)

Summary notes on what the model could achieve:

• pull out the code required to run a http server into the grpc plugin
• enable high performance ipc
• mechanism for versioning the api
• abstract away the CLI from the actual library. see (ctr) in containerd, which is a "reference" implementation of how to consume the gRPC server.

Furthermore, gRPC is included in the CNCF alongside rkt, kubernetes and more...

Would be happy to help with this