Create webgw

[LeeSmet]

Goal

We intend to create a set of networking tools for use on the grid, which we will call "webgw". The main idea is to create an application offering a variety of functions to the users. Nodes which have public configs will run a "server" binary, while users/hidden nodes run a "client" binary. Configuration (when needed) is intended to be done through a to be developed tendermint based blockchain. We aim for following functionality:

• Tcp proxy with connection sniffing such that we can offer multi tenant proxying from public nodes to hidden nodes (similar to tcprouter). Note that we don't intend to terminate TLS.
• Websocket proxy support, although given how websocket works this should be accomplished by the previous point.
• DNS support (at least on public nodes). This will be an option both for "internal" grid use (e.g. microservices each being identified by their own name), but for those interested, they should also be able to point their domains NS to these grid DNS servers.
• Overlay network based on wireguard. The idea here: all public nodes are interconnected, and run the aforementioned tendermint based blockchain. This chain creates a consensus between all nodes on how the topology of the network should look like. A "hidden" node connects to some public nodes (say 2), which then store this in the chain. This means all nodes agree on network topology, all public nodes (nodes running the chain) know how to reach every other node, and thus all hidden nodes can thus send their packets directly to a public node, and all nodes are reachable through at most 2 hops.

Of course, the aim is to create this in such a way that we achieve good scalability.

It is agreed that for now we will focus on delivering only the first of these items, while keeping in mind that the other features will be implemented in the future.

Work to be done

• [x] Create and setup new repo
• [x] Create initial spec, keeping in mind the extensibility as described above
• [x] Initial implementation of tcp/web proxy
• [x] Documentation

Technical issues

• https://github.com/threefoldtech/webgw/issues/1
• https://github.com/threefoldtech/webgw/issues/2
• https://github.com/threefoldtech/webgw/issues/3
• https://github.com/threefoldtech/webgw/issues/4
• https://github.com/threefoldtech/webgw/issues/5
• https://github.com/threefoldtech/webgw/issues/6
• https://github.com/threefoldtech/webgw/issues/7

[sabrinasadik]

@despiegk pls give your ok for this

[despiegk]

its not clear to me what we have now, can we make example in the builder directory with docker compose, the goal doesn't seem to be in line to what we discussed

[LeeSmet]

As agreed for now we implemented web proxy part.

Hash function has been changed from blake2b to SHA256, as the former is not available in V at this time.

[LeeSmet]

Initial code is done, including change requested. Pre release is made on the repo which includes pre compiled linux binaries.

Current implementation now needs to be load tested.

[xmonader]

while this won't be part of 3.9 verification, i'll move to 3.10 in verification status.

[mohamedamer453]

Verified.

All of the tasks mentioned in the issue were completed.

Work to be done

• [x] Create and setup new repo
• [x] Create initial spec, keeping in mind the extensibility as described above
• [x] Initial implementation of tcp/web proxy
• [x] Documentation

• The repo was created and available at https://github.com/threefoldtech/webgw.
• A descriptive readme with startup example was added in the repo home. more docs about the usage and architecture added in the docs directory.
• Initial implementation was done and i was able to test the client/server setup mentioned in the readme docs without any issues.