Building and operating back-end apps, such as public endpoints and always-on private endpoints, is hard:
Developers are constrained to the languages/platforms we support (i.e., Node.js and the JVM), so they can't use Python, Ruby, etc., until RAMF, PoHTTP, the PKI and the Channel Session Protocol become available on those platforms.
Developers and operators are responsible for key management, which is hard to do well and expensive to get wrong -- although keystore libraries like keystore-vault-js do help to some extent.
Due to the use of DNSSEC and SRV records, developers can't do E2E testing locally: all changes must be deployed to an Internet host and then tested there. This is especially bad for newcomers as they need to do a lot of groundwork before they get to the fun stuff.
The solution
We should build a multi-tenant middleware app that sits between gateways and back-end apps.
This middleware would expose a PoHTTP server to receive parcels from gateways. When a new parcel is received, it'd be unwrapped and its service message sent to the backend in the local network or on the Internet.
Whenever the backend wishes to send a parcel, it'd just pass the service message to the middleware which will then wrap the service message in a parcel and send to the respective public gateway.
The backend and the middleware will communicate via HTTP only initially, and methods like GCP PubSub, NATS and Kafka will be supported in the future (maybe using CloudEvents).
To ensure authenticity and integrity, and avoid replay attacks, every message between the backend and the middleware MUST be signed. The current time should be included in the signature plaintext, and its verification should be tolerant to clock drifts in the range of +/- 5 minutes. Public keys would be exchanged as part of a pairing process that kicks off when a new backend/endpoint is registered in the middleware: The middleware posts its public key to the backend, and the backend then posts its public key along with a digest of that of the middleware.
This middleware will use Vault to manage sensitive data, such as the private component of identity keys.
App developers will choose whether service messages are decrypted for them (analogous to TLS termination) or not (to keep the E2E encryption). The former makes things drastically easier for them but the latter offers much better security/privacy. When they choose to keep E2EE, they'll be responsible for using Vault or a similar solution to keep ephemeral encryption keys secure.
With a system like this in place, it'd be fairly easy to make this middleware act as a private gateway so that we can trigger webhooks/etc in response to incoming messages bound for backends.
Describe alternatives you've considered
The following will only partly solve the problems above:
Using Kotlin nativemight make it possible to support many platforms, so it'd partly solve the x-platform issue. We'd still have to write Python/Ruby/etc libraries that use the native output.
Implementing a high-level endpoint framework for each language/platform (a la awaladroid) would help with the key management issue.
gnarea
commented
1 year ago
The problem
Building and operating back-end apps, such as public endpoints and always-on private endpoints, is hard:
The solution
We should build a multi-tenant middleware app that sits between gateways and back-end apps.
This middleware would expose a PoHTTP server to receive parcels from gateways. When a new parcel is received, it'd be unwrapped and its service message sent to the backend in the local network or on the Internet.
Whenever the backend wishes to send a parcel, it'd just pass the service message to the middleware which will then wrap the service message in a parcel and send to the respective public gateway.
The backend and the middleware will communicate via HTTP only initially, and methods like GCP PubSub, NATS and Kafka will be supported in the future (maybe using CloudEvents).
To ensure authenticity and integrity, and avoid replay attacks, every message between the backend and the middleware MUST be signed. The current time should be included in the signature plaintext, and its verification should be tolerant to clock drifts in the range of +/- 5 minutes. Public keys would be exchanged as part of a pairing process that kicks off when a new backend/endpoint is registered in the middleware: The middleware posts its public key to the backend, and the backend then posts its public key along with a digest of that of the middleware.
This middleware will use Vault to manage sensitive data, such as the private component of identity keys.
App developers will choose whether service messages are decrypted for them (analogous to TLS termination) or not (to keep the E2E encryption). The former makes things drastically easier for them but the latter offers much better security/privacy. When they choose to keep E2EE, they'll be responsible for using Vault or a similar solution to keep ephemeral encryption keys secure.
With a system like this in place, it'd be fairly easy to make this middleware act as a private gateway so that we can trigger webhooks/etc in response to incoming messages bound for backends.
Describe alternatives you've considered
The following will only partly solve the problems above: