Enable cloud agnostic containerization and horizontal scaling

[zacoppotamus]

Some ideas for accomplishing this while also attaining dev/prod parity (h/t @slowr for the suggestions):

• Utilize a cloud provider's blob storage functionality for storing the DistilBert model and ES index, or alternatively investigate using blob storage in PostgreSQL;
• Transition to Kubernetes and package the application as a Helm package;
• Use Terraform for infrastructure provisioning;
• Use an Elasticsearch image in docker-compose as an alternative to spinning up a dedicated ES service;
• Handle DNS management through Terraform, rather than manually through Namecheap;
• Investigate Localstack.

[kstathou]

These are all great suggestions, thank you @slowr! Using terraform and kubernetes would be beneficial (and a technical challenge we'd like to work on!).

Would it make sense to run kubernetes (AWS EKS) on Fargate? Some thoughts on our use case:

• Our pipeline needs to run every forthnight - we don't need the system to be always up.
• Orion is still in its early days so it might be better to go serverless and focus more on client-facing features. Fargate would deal with some of the complexity of running kubernetes (autoscaling, IAM identity management) and maybe reduce the overall cost of the system (see here and here)

Would this be a sensible setup? Happy to learn about alternatives too!

[slowr]

For your use-case going server-less and using AWS EKS with Fargate (or GCP Cloud Run, etc) is the best solution price-wise and I would choose that for sure.

Fargate removes the need of specifying how many nodes you need on the control-plane when setting up the cluster which is a great resource-elasticity feature.

Be very careful on your CPU and RAM resource limits defined in your kubernetes manifests as Fargate uses them as source-of-truth to allocate the machines and do the pricing.

[kstathou]

That's great, thanks a lot for the suggestions and help!