Replace kfdefs with DataScienceCluster resource and changer default ns to opendatahub for make deploy

[HumairAK]

With ODH v2 now on the fast channel, it's time we remove mentions of kfdefs, and replace them with DataScienceCluster resource. The resources should be included in their own sub folder like we had/have with kfdef here.

Provide typical DSC example with just dashboard + datasciencepipelines, a README on how to use it, and also a DEV example utilizing the manifests field, here. The example should include one pointing to the main branch with inline comments on suggestions for tags/etc. Now that kfdef parameters are gone, how can we deploy a custom DSPO image with v2 operator, like when we are testing a PR, is there a way? if so provide suggestions within the readme.

Also it seems odh v2 operator now deploys apps to opendatahub namespace, confirm this your self first. Once confirmed, update the make deploy command to deploy to this namespace by default instead of odh-applications.

[HumairAK]

here's an example DSC I used:

kind: DataScienceCluster
apiVersion: datasciencecluster.opendatahub.io/v1
metadata:
  name: default
spec:
  components:
    dashboard:
      managementState: Managed
    datasciencepipelines:
      devFlags:
        manifests:
          - uri: https://github.com/opendatahub-io/data-science-pipelines-operator/tarball/main
            contextDir: config
            sourcePath: base
      managementState: Managed

[hbelmiro]

@HumairAK it looks like this issue depends on https://github.com/opendatahub-io/opendatahub-operator/issues/673.

[HumairAK]

@hbelmiro in this case we can leave the dev example out but do the rest, let's do it this way:

• Provide the DSC for end user
• Provide a work around for dev to deploy a DSC deploying from a fork/branch, with list of commands to scale down odh operator and patch in the deployment image with a dev image
• Create a follow up issue on updating this workaround with proper image fields once implemented in odh operator (mark this blocked, link to the blocking issue, and also mark it good-first-issue)