[POC] Catalog Views and integration with Service Manager

[PK85]

Description

Provide new UI for BTP Operator in the Busola where users can see Service Catalog. For now it could be separate application, if installed in the Kyma cluster should already work. In the future probably it will be a part of the btp-manager module.

Reason

We want to make BTP Operator user's lives easier and more convenient where they can see all of the available services in the Busola and then just provision some of them.

AC

• [x] 1) Direct integration with the Service Manager

  • [x] 1.1) User can use default credentials + can select secret from any namespace (SM credentials are from different SA and then could see different instances and service offerings) https://github.com/kyma-project/btp-manager/pull/655 https://github.com/kyma-project/btp-manager/pull/674 https://github.com/kyma-project/btp-manager/pull/703
  • [x] 1.2) user can see a list of all service offerings from Service Manager from given SA (SM credentials are bound to given SA) https://github.com/kyma-project/btp-manager/pull/717
  • [x] 1.3) User can see a list of all existing instances from Service Manager from given SA (SM credentials are bound to given SA) https://github.com/kyma-project/btp-manager/pull/742, https://github.com/kyma-project/btp-manager/pull/735, https://github.com/kyma-project/btp-manager/pull/746 https://github.com/kyma-project/btp-manager/pull/758
  • [x] 1.4) User can create a service binding on Service Manager side which results as a secret in the selected namespace in the Kyma environment https://github.com/kyma-project/btp-manager/pull/735 https://github.com/kyma-project/btp-manager/pull/757
  • [x] 1.5) User can delete a service binding on Service Manager side which invalidates given secret (probably we need to delete corresponding secret) https://github.com/kyma-project/btp-manager/pull/757 https://github.com/kyma-project/btp-manager/pull/772

• [ ] 2) New BTP Operator UI in the Busola

  • [ ] 2.1) check with Busola team, on how to embed our own web page (We should have a view in the namespace + in the overview taking into account all namespaces data)

• [ ] Think in future of making it mobile - friendly

Links

• check smctl what calls are executed
• https://github.com/Peripli/service-manager
• https://help.sap.com/docs/service-manager/sap-service-manager/sap-service-manager

[PK85]

Reference sharing:

apiVersion: services.cloud.sap.com/v1
kind: ServiceInstance
metadata:
  name: kyma-demo-instance-123
  namespace: develope
spec:
  serviceOfferingName: xsuaa
  servicePlanName: reference-instance
  parameters:
    referenced_instance_id: <shared service id>
---
apiVersion: services.cloud.sap.com/v1
kind: ServiceBinding
metadata:
  name: kyma-demo-binding-123
  namespace: develope
spec:
  serviceInstanceName: kyma-demo-instance-123

[kyma-bot]

This issue or PR has been automatically marked as stale due to the lack of recent activity. Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Close this issue or PR with /close

If you think that I work incorrectly, kindly raise an issue with the problem.

/lifecycle stale

[kyma-bot]

This issue or PR has been automatically closed due to the lack of activity. Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle stale

If you think that I work incorrectly, kindly raise an issue with the problem.

/close

[kyma-bot]

@kyma-bot: Closing this issue.

In response to [this](https://github.com/kyma-project/btp-manager/issues/442#issuecomment-1888786355): >This issue or PR has been automatically closed due to the lack of activity. >Thank you for your contributions. > >This bot triages issues and PRs according to the following rules: >- After 60d of inactivity, `lifecycle/stale` is applied >- After 7d of inactivity since `lifecycle/stale` was applied, the issue is closed > >You can: >- Reopen this issue or PR with `/reopen` >- Mark this issue or PR as fresh with `/remove-lifecycle stale` > >If you think that I work incorrectly, kindly [raise an issue](https://github.com/kyma-project/test-infra/issues/new/choose) with the problem. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.

[ukff]

Issue with UI development for embedding pages: https://github.com/kyma-project/busola/issues/2762

[szwedm]

Meeting notes and current arrangements (3.04.24)

Arrangements:

• support operations as in Service Manager
• create REST API similar to Service Manager API (OSBAPI)
• allow to connect Service Instances and Service Bindings from Service Manager to the cluster (custom resources in the cluster should reflect instances and bindings in SM)

Limitations:

• Service Manager context is limited to the scope of SAP BTP service operator client provided during Kyma environment provisioning

Future ideas:

• allow update of the cluster ID
• service management as a BTP user (subaccount context)

[ukff]

early proposal of UI:

[ukff]

[szwedm]

SAP BTP service operator recognises the following secrets for service instances:

• default secret named sap-btp-service-operator in the centrally managed namespace (kyma-system)
• secret for a different subaccount mapped to a namespace in the cluster with namespace's name as a prefix in the secret's name <namespace-name>-sap-btp-service-operator in the centrally managed namespace (kyma-system)
• secrets named sap-btp-service-operator in other namespaces mapped to different subaccounts
• arbitrarily named secret in the centrally managed namespace (kyma-system) connected to a different subaccount pointed in the ServiceInstance btpAccessCredentialsSecret field in the spec

[szwedm]

PR introducing SecretProvider which fetches secrets with credentials for Service Manager: https://github.com/kyma-project/btp-manager/pull/655

[szwedm]

UI PR: https://github.com/kyma-project/btp-manager/pull/649

[szwedm]

Adding Service Manager client: https://github.com/kyma-project/btp-manager/pull/674

[szwedm]

Unit test for SM client service offerings: https://github.com/kyma-project/btp-manager/pull/703

[ukff]

[ukff]

Sprint review comments (21.05.2024):

• Marketplace: ServiceOffering details + plans details
• Marketplace: Search in UI
• Marketplace: Create (PUT) Service Instances
• SI: get number of instances for given secret
  • Indicator is it on SM/Kyma or both (Postpone for next phase)
• UI: Show error image
• UI: Show loading image
• Layout:
  • reduce grey spaces
  • make more wider
  • make biggest card

[ukff]

Meeting notes 04.06.24

1: Service Manager resources (frontend needs backend first)

• Marketplace: ServiceOffering details + plans details Backend (Marcin)
• Marketplace: ServiceOffering details + plans details Frontend (Lukasz)
• Marketplace: Create CRUD Service Instances Backend (Marcin)
• Marketplace: Create (PUT) Service Instances Frontend (Lukasz)
• SI: get service instances for given secret from SM Backend (Marcin)
• SI: get service instances for given secret from SM Frontend (Lukasz)

2: Independent from backend:

• Marketplace: Search in UI
• UI: Show error image
• UI: Show loading image
• Layout:
  • reduce grey spaces
  • make more wider
  • make biggest card

3:

• Security to investigate how to restrict access to secrets for given users
• Bundle backend and frontend without port conflicts (Marcin)
  • Decide how to run everything together in 1 pod

[szwedm]

• Marketplace: ServiceOffering details + plans details Backend (Marcin)

Done in https://github.com/kyma-project/btp-manager/pull/717

[szwedm]

• Bundle backend and frontend without port conflicts (Marcin)

  • Decide how to run everything together in 1 pod

Done in: https://github.com/kyma-project/btp-manager/pull/720

[ukff]

UI: https://github.com/kyma-project/btp-manager/pull/728

[szwedm]

• Marketplace: Create CRUD Service Instances Backend (Marcin)

Done in: https://github.com/kyma-project/btp-manager/pull/742

[szwedm]

Service Bindings in SM client and Service Instances in API: https://github.com/kyma-project/btp-manager/pull/735

[szwedm]

To do (as of 16.07.24):

• API: create, update, delete Service Instances @szwedm https://github.com/kyma-project/btp-manager/pull/758
• API: list, delete Service Bindings @KsaweryZietara https://github.com/kyma-project/btp-manager/pull/757
• API: tests @szwedm
• Frontend: Service Instances @ralikio
• Frontend: Service Bindings @ralikio

[szwedm]

Service Bindings API: https://github.com/kyma-project/btp-manager/pull/757

[szwedm]

Service Instances API: https://github.com/kyma-project/btp-manager/pull/758

[szwedm]

Service Bindings API extension and tests: https://github.com/kyma-project/btp-manager/pull/772

[szwedm]

Meeting notes 31.07

• [x] 1) allow to set secret's name and namescape, https://github.com/kyma-project/btp-manager/pull/794 https://github.com/kyma-project/btp-manager/pull/798 https://github.com/kyma-project/btp-manager/pull/798
• [x] 2) if secret's name is missing, use name from Service Binding. Allow user to provide proposed default value for secret's name (e.g. SB's name + random characters), https://github.com/kyma-project/btp-manager/pull/794 https://github.com/kyma-project/btp-manager/pull/798 https://github.com/kyma-project/btp-manager/pull/798
• [x] 3) send empty values for clusterid, namespace, k8sname when creating SI/SB,
• [x] 4) display the available secret with its name for Service Binding. If the secret is missing, let the user know and allow to create it in any namespace, https://github.com/kyma-project/btp-manager/pull/802 https://github.com/kyma-project/btp-manager/pull/843 https://github.com/kyma-project/btp-manager/pull/854
• [X] 5) document questions to CIS team whether there is a way to update context (clusterid, namespace) for existing SIs and SBs (apparently it looks like created SI in K8s cluster cannot change the namespace without deletion of actual service instance).

[szwedm]

Error handling improvements: https://github.com/kyma-project/btp-manager/pull/779

[PK85]

Meeting notes 6.08.2024:

• [x] 1) UI: Make Top Bar similar to busola one, like kubeconfig selection. https://github.com/kyma-project/btp-manager/pull/797
• [x] 2) UI: When Marketplace or Service Instance selected, display on right top a bar with for MarketPlace: Service Offerings for Service Instances keep the same name - @ralikio - https://github.com/kyma-project/btp-manager/pull/817 (title added on the left side)
• [x] 3) UI: width for secret selection must be much wider, we have a lot of space there https://github.com/kyma-project/btp-manager/pull/797 - @ralikio changed selection to be based on menu items to be more in touch with ui5 look and feel and make migration easier.
• [x] 4) UI: I added new secret in the k8s, then when clicking secret combobox nothing happen. Looks it is loaded only at the beginning once whole page is rendered #809
• [x] 6) UI: if list secrets return error or empty,
  • [x] show only secret seletion view. Nothing more should be displayed not to have another failures - case when loaded secret does not contain valid data - not able to connect https://github.com/kyma-project/btp-manager/pull/825
  • [x] display proper message (errro occured, or no secrets to display). Ask user to click refresh button. https://github.com/kyma-project/btp-manager/pull/820
• [x] 7) UI: when adding label to SI and SB, give information that "=" is a separator and user must click enter to add a key #840
• [x] 8) UI:
  • [x] When deleting Binding, it is still on the list, https://github.com/kyma-project/btp-manager/pull/803
  • [x] and when refreshing whole page, 404 is displayed - https://github.com/kyma-project/btp-manager/pull/830
• [x] 9) UI: when calling localhost:8080/instances, http://localhost:8080/offerings, http://localhost:8080/bindings then 404. It should be loaded - https://github.com/kyma-project/btp-manager/pull/792
• [x] 10) UI + REST: All Rest APIs where secret is needed to execute a job, secret name must be provided and cannot be a central one like now. Providing secret is a mandatory action, remove central cache with secret, do not fallback.
  • [x] 10.1) introduce to params: secretName and secretNamespace, both required #844
  • [x] 10.2) Pass secret name and namespace to every api endpoint @ralikio
• [x] 11) when users selects details view dipslay that the same like in the Busola, use column layout - https://github.com/kyma-project/btp-manager/pull/850

[ralikio]

I have discovered a troublesome behaviour: If a service binding with the same name binding (just an example name that does not matter in the example) for both binding and secret exists then creating another service binding with the following request fails but binding is created despite the API error:

{
name: "binding-new-name"
secret_name: "binding"
secret_namespace: "default"
service_instance_id: "872fe846-569e-4e54-98de-04983e8ea194"
}

[szwedm]

I have discovered a troublesome behaviour: If a service binding with the same name binding (just an example name that does not matter in the example) for both binding and secret exists then creating another service binding with the following request fails but binding is created despite the API error:

{
name: "binding-new-name"
secret_name: "binding"
secret_namespace: "default"
service_instance_id: "872fe846-569e-4e54-98de-04983e8ea194"
}

PR with the fix: https://github.com/kyma-project/btp-manager/pull/826

[PK85]

Testing results:

• [x] When adding new service binding, both name and secret name should have random suffix added. Right now only Secret name has it, see: @MarekMichali
• [x] I added SB and secret, then kubectl delete secret, then back to UI. I was able to recreate secret. Then I changes SI to another one, and I was only able to change Secret Name. Binding name field was disabled for all instances. Even wrong Secret Name is suggested, see: @MarekMichali Hint: when I go to marketplace and back, it works again.
• [x] When recreating secret for given Service Binding, suggest Secret Name with the same name as binding (do not add new suffix for Secret Name) @MarekMichali
• [x] Still no way to close error windows, see: @ralikio https://github.com/kyma-project/btp-manager/pull/857

[PK85]

How to demo: 1) show 3 credentials in the Kyma cluster, two from SA_1 and one from SA_2 (show new UI, Busola and BTP Cockpit)

• 1.1) we use SM instance with 'service-operator-access' plan designed for SAP BTP Service Operator 2) add new entitlement to one of the SA and show new UI that marketplace is extended with new offering (classes and plans) 3) show how to create SI
• 3.1) we pass empty required "namespace" parameter when using SM 'service-operator-access' . Looks that namespace cannot be changed later on. 4) show how to create SB and secret in the given namespace 5) explain how thet looks like in the BTP Cockpit Service instances view (how to find which SM instance was used) 6) show how to restore a secret for given SB 7) Show SI created by SAP BTP Service Operator in our new UI and BTP Cockpit 8) Say what next:
• 7.1) How to bundle that into busola + authentication (https://github.com/kyma-project/busola/issues/3122)
• 7.2) Implement offerings view in the busola

Note: right now we do not pass CLUSTER_ID. Then in the BTP Cockpit instance we see SM instance name-serviceID