Change manifest API (should to be more consistent)

vlad-tokarev commented 3 years ago

training example

id: wine
kind: ModelTraining
spec:
  model:
    name: wine
    version: 1.0
  toolchain: mlflow
  entrypoint: main
  workDir: mlflow/sklearn/wine
  # READ - ONLY
  featureSource:
    - connName: features-source #(bucket: odahu-demo01-feature-store)
      localPath: wpm-model/rc01/
      namespace: ...
  # READ - ONLY
  algorithmSource:
    connName: algorithms-source
  # WRITE - ONLY
  modelRegistry:
    connName: models-registry
  hyperParameters:
    alpha: "1.0"
  resources:
    limits:
      cpu: 4
      memory: 4Gi
    requests:
      cpu: 2
      memory: 2Gi

Packaging

id: wine
kind: ModelPackaging
spec:
  modelsRegistry:
    connName: models-registry
    modelName: "<fill-in>"  # (path / archive name inside bucket)
  serviceRegistry:
    type: docker
    pull-connection: private-gcr-registry
    push-connection: docker-public-registry
#  serviceRegistry:
#    type: cloud-function
#    bucket-connection: cloud-function-bucket-connection
  integrationName: docker-rest
  resources:
    limits:
      cpu: "1.5"
      memory: 4Gi
    requests:
      cpu: "1.5"
      memory: 2Gi

ModelDeployment -> InferenceService

id: wine
kind: InferenceService
spec:
  serviceRegistry:
    type: docker
    pullConnection: private-gcr-registry
    imageName: model-1
  modelsRegistry:
    connName: models-registry
    modelName: "<fill-in>"  # (path / archive name inside bucket)
  predictor: odahu-ml-server
  minReplicas: 1

BatchInferenceService

id: wine
kind: BatchInfernceService
spec:
  serviceRegistry:
    type: docker
    pullConnection: private-gcr-registry
    imageName: generic-mlflow-predictor
  modelsRegistry:
    connName: models-registry
    modelName: "<fill-in>"  # (path / archive name inside bucket)

karbyshevds commented 3 years ago

As a result connections (output, input, etc) should be customizable via manifests described above.

alinaignatiuk commented 3 years ago

Check with D.S. the status and validate how it should be done right now. Whether we can leverage the examples above

odahu / odahu-flow

Change manifest API (should to be more consistent) #541