Fractl as a distributed compute platform

[vijayfractl]

Currently remote resolution of patterns are enabled by a configuration - this should happen automatically. For example, if the pattern {:Acme/Employee {...}} cannot be resolved to a local entity, the pattern should trigger a remote-request. The fractl-platform may run a "root-url-resolution-service" which can be used by the runtime to resolve the root-url for the :Acme model. This service also provide information on the type of the pattern (here :Acme/Employee will be identified as an :entity), so the appropriate remote-request can be created (eval vs upsert). With the ability to automatically execute remote requests, fractl apps can collaborate in an intrinsically distributed environment.

[vijayfractl]

Some notes on deployment in general - A fractl application executes on a "runtime" comprising of the following components:

1. The storage layer (by default postgres with replication and backup)
2. A distributed caching layer (TBD)
3. Auth-service (by default a keycloak cluster)
4. Model-naming service (TBD - for auto-resolving remote calls)
5. Model repository (TBD)
6. Reporting & monitoring components - filebeat, kibana
7. HTTP gateway for load-balancing

A fractl application (i.e fractl jar + model) runs on multiple instances fronted by the HTTP gateway. For a dev-deployment a shared runtime (1-7) with some logical partitions can be used. For instance, a database "schema" or a keycloak "realm" can be thought of as a logical partition for testing an app.

For production deployment, a dedicated "runtime" should be created. An instance of the runtime should be described as a fractl model, which can be auto-inferred from the app-model's config and spec. I think the "runtime" model (which describes the runtime-topology of the app) should be used in combination with a resolver for some generic tool like Terraform - so we don't have to worry about the details of each PaaS.

With the distributed nature of resolving patterns (4), model-dependencies need not be packaged with the app. The deployer can "check" if the correct version of the dependency-model is deployed and running - if not signal an error. If a model "repository" is part of the platform, dependencies can be deployed as needed.

[fractlrao]

@vmatv8, would it be possible to support two different deployment models:

1. Serverless - needed in the short-term, as we can scale out at lower cost as we onboard the initial surge of number of users
2. Compute platform with containers - needed for production applications. Can be lower priority for the immediate future

For the serverless deployment model, can we leverage the cloud-native services, so that we don't have to manage the infrastructure?

[vijayfractl]

Related issues:

https://github.com/fractl-io/fractl-platform/issues/3

[vijayfractl]

@fractlrao I think the following issues should be completed in the given order to get deployment of apps working:

1. 677

2. 679 - depends on (1), this is a "bootstrap" service and should be manually deployed

3. 678 (it should now be possible to auto-deploy this using the results of (1) and (2)

4. 680