search

datarevenue-berlin / OpenMLOps

MIT License
703 stars 101 forks source link

Question: Do I need to learn Terraform, Kubernetes, AWS services? #84

Closed offchan42 closed 2 years ago

offchan42 commented 3 years ago

Ideally everything should be abstracted away but I know that's impossible.

  1. Do I need to learn what Terraform does? E.g. by taking its tutorial
  2. Kubernetes?
  3. AWS services? Will this thing be abstracted from me if there's error happening?

I assume that things like prefect, MLFlow, Dask, Feast, Seldon cannot be abstracted away. I mean I need to learn them from their tutorials in order to use this repo. Which I find acceptable. But I really don't want to dive into Terraform, Kubernetes, and especially AWS services as it seems there are lots of things to learn and it could be a big rabbit hole. Am I understanding this correctly?

To ask it differently, what are the basic knowledge requirements needed to use this repo (and fix bugs when it happens)?

bernardolk commented 2 years ago

Hello @off99555 - I understand your situation: Ideally, yes, the infrastructure would be abstracted away for you and you would have only to deal with the higher-level services as you described. In practice, every deployment situation is a bit different, so knowledge of Terraform, EKS (AWS), and Kubernetes would be necessary to make it work for your specific deployment case. Even if we did a perfect job on our Terraform configuration, you would still probably have issues caused by your own machine's environment, but that's the direction we want to move - make it as easy as possible for you to use OpenMLOps without needing to know what's under the hood.