datarevenue-berlin / OpenMLOps

MIT License
695 stars 101 forks source link

Update the tutorial Creating a basic machine learning system on Open MLOps with sample values from local minikube #92

Open vfortu opened 2 years ago

vfortu commented 2 years ago

it would be great to have also the default values of url and domains when setting up in a local environment with the defaults my_vars.tfvars defined in the minikube guide. So for example:

how these aws = false db_username = "mlflow-db-user" db_password = "mlflow-db-pasword" hostname = "myambassador.com" ory_kratos_cookie_secret = "secret" ory_kratos_db_password = "password" install_metrics_server = false install_feast = false install_seldon = false prefect_create_tenant_enabled = false jhub_proxy_secret_token = "IfYouDecideToUseJhubProxyYouShouldChangeThisValueToARandomString" enable_ory_authentication = false oauth2_providers = [] mlflow_artifact_root = "/tmp"

match with domain = "mlops.ritza-route53.com" # the domain where you are hosting Open MLOps username = "gareth@ritza.co" # the username you used to register on Open MLOps password = "DkguP5GsB9yiPk8" # the password you used to register on Open MLOps s3_bucket = "another-mlops-bucket" # the S3 bucket you specified when setting up Open MLOps

prefect_project_name = "wine-quality-project"         # you can use what you want here
docker_image = "drtools/prefect:wine-classifier-3"    # any docker image that has the required Python dependencies

or also how to reach in the minikube case: => https://mlops.example.com/profile/auth/registration
=> https://jupyter.mlops.example.com

that would be great thanks!

pipatth commented 2 years ago

Hi @vfortu,

Thank you for your input. I created a pull request #93 with a note on which variables in myvar.tfvars can be changed by the users.

About accessing the MLOP services, please follow this section: https://github.com/datarevenue-berlin/OpenMLOps/blob/master/tutorials/set-up-minikube-cluster.md#trying-out-the-services

Because the minikube deployment doesn't have a authentication system, you don't have to register a user to use the services. Just port-forward the service you want to localhost and use it :)

vfortu commented 2 years ago

thank you so much for the prompt feedback.

got it!

but I still have some issues in the prefect part since it seems to not be able to connect with prefect_url = f"https://prefect.{domain}/graphql" being the domain = "myambassadorcom" # the domain where you are hosting Open MLOps, that I've left as the default suggested in the minikube

any clues?

vfortu commented 2 years ago

In particular I've issue when running the "create_prefect_flow()" so it is clear that there is a mismatch in the settings of : domain = "me.com" # the domain where hosting Open MLOps username = "me-user" # the username used to register on Open MLOps password = "me-pasword" # the password used to register on Open MLOps s3_bucket = "s3://mlops-model-artifact" # the default gathered from the variables.tf file prefect_project_name = "quality-project" # free field docker_image = "drtools/prefect:wine-classifier-3" # any docker image that has the required Python dependencies

since it seems that in the prefect dashboard page, the "open the connection menu" button in the high-right part is red and signals that couldn't connect to => http://prefect.me.com/graphql

pipatth commented 2 years ago

Hi @vfortu,

Sorry for the bug. This is fixed in https://github.com/datarevenue-berlin/OpenMLOps/pull/99

Please be aware that the instructions are a little different (different myvars.tfvars, port-forward). Let me know if this fix works for you. Thank you.

sramirez commented 2 years ago

Hi all,

Actually, if you install the stack without ory authentication things get more complicated when configuring prefect connection. I tried with different configurations but no success till now. Would it possible to replace the ipynb file with a more basic example about the Prefect part?

bernardolk commented 2 years ago

Hi all,

Actually, if you install the stack without ory authentication things get more complicated when configuring prefect connection. I tried with different configurations but no success till now. Would it possible to replace the ipynb file with a more basic example about the Prefect part?

Hello. If I understand correctly you are suggesting that we don't use Ory in the minikube tutorial/setup, right?

sramirez commented 2 years ago

Yes, maybe in this way the minikube tutorial will be closer to a real scenario with local deployment w/o authentication.

I tried to connect with the following sentences:

prefect_url = f"http://IP:4200/graphql"
prefect_client = Client(api_server=prefect_url)

I managed to launch an scheduled job in Prefect, but at the same time getting errors with the command. On the other hand, it was not possible the creation of projects from python (I did it manually from the UI).