ARM64 release and image

[jiayihu]

Hi, I see standard amd64 targets are available. Would be possibile to add support to ARM64/aarch64?

[josephlewis42]

Hi @jiayihu, I think it might be possible, but I'm not sure there's enough demand to officially support it. The broker is mostly used with Cloud Foundry installations, which I don't think support ARM64/aarch64 targets.

You could try building the Dockerfile yourself or using go build with main.go. If you're starting a net-new application, I'd recommend using the cloud-service-broker instead of the gcp-service-broker. It's a fork of this broker that just keeps the brokerpak specification (I gave a talk about it here) which uses Terraform templates to provision resources across multiple cloud environments.

[jiayihu]

I was right now able to make it work by using the darwin binary and registering within the local k8s cluster using svcat. Unfortunately I don't know Terraform yet, at a first looks it sounds like AWS Cloudformation generalized for every provider.

I also watched your lightning talk and my understanding is that cloud-service-broker doesn't really suit my needs if I understood correctly. I don't need to support multiple providers (for now) nor to allow users to provision services by using templates. I'm just working on my master thesis 😄

I'm building a k8s cluster of raspberry pi running WASM kubelets (via krustlet) along with Rust embedded IoT devices, which also are able to run small WASM modules. I'd like to expose the cluster WASM computing capabilities, IoT device informations and GCP resources as services by composing different brokers. For this needs I don't think I need Terraform, please correct me if I am wrong.

That being said, I hope I can get some time to learn Go and make a PR to this repo to support cross-compiling to ARM64 if you are okay with it.

[josephlewis42]

Unfortunately I don't know Terraform yet, at a first looks it sounds like AWS Cloudformation generalized for every provider.

That's about right, Terraform is an infrastructure management tool that can be used with dozens of different "providers" to provision infrastructure. The benefit is that you can abstract out what "infrastructure" means to you. The GCP Service Broker was built with Cloud Foundry in mind which has benefits and drawbacks. The benefit is that each item is standalone, but the drawback is that the plans are often very restrictive and limited.

If your solution is Kubernetes native and doesn't require the provision/bind semantics that OSB provides, you could also look at config-connector which is Google's way of exposing infrastructure in the K8s cluster as custom resources. AWS has a similar solution.

I'm building a k8s cluster of raspberry pi running WASM kubelets (via krustlet) along with Rust embedded IoT devices, which also are able to run small WASM modules. I'd like to expose the cluster WASM computing capabilities, IoT device informations and GCP resources as services by composing different brokers. For this needs I don't think I need Terraform, please correct me if I am wrong.

Interesting, is your intent to use Kubernetes as the control plane where you bind in new IoT devices as kubelets and connect them to new or existing infrastructure? If that's the case, I would probably try to go with using config-connector (or the gcp-service-broker). If each IoT device needed some suite of things (e.g. a Secret, a Pub/Sub connection, a service account, etc.) you might be able to do the composition you need using minibroker; it's an OSB compatible broker that runs Helm scripts against the k8s cluster it's installed in.

That being said, I hope I can get some time to learn Go and make a PR to this repo to support cross-compiling to ARM64 if you are okay with it.

I'd be happy to take a PR. Cross-compiling in go is pretty straightforward, it looks like arm64 would work, but not wasm:

$ GOOS=linux GOARCH=arm64 go build -o gcp-service-broker-arm ./main.go 

$ GOOS=js GOARCH=wasm go build -o gcp-service-broker-wasm ./main.go 
# github.com/GoogleCloudPlatform/gcp-service-broker/vendor/github.com/spf13/viper
vendor/github.com/spf13/viper/viper.go:281:19: undefined: fsnotify.NewWatcher

Best of luck with your thesis!

[jiayihu]

If your solution is Kubernetes native and doesn't require the provision/bind semantics that OSB provides, you could also look at config-connector which is Google's way of exposing infrastructure in the K8s cluster as custom resources. AWS has a similar solution.

Unfortunately I need the binding semantics, that's why I excluded ConfigConnector. Actually so far service catalog seems to be meant for people to easily manually provision pre-defined services in a cluster. In my case I'm actually planning on writing a service API on top of the different service brokers to allow any application to ask for services themselves in a machine-to-machine communication. The idea is that the application can be deployed on edge and ask for services. Then the cluster can offer different services and plans based on what it can offer. For instance if you need a database, I can offer you mariadb in GCP or locally but it will cost you differently.

Interesting, is your intent to use Kubernetes as the control plane where you bind in new IoT devices as kubelets and connect them to new or existing infrastructure?

Exactly, but I'm more keen on using akri to expose IoT devices via device plugin and allow applications to connect to them via an intermediate broker. The latter broker is more a messaging broker with resource discovery mechanism, not a service broker, and it should allow other pods to use the device as a K8s service.

If each IoT device needed some suite of things (e.g. a Secret, a Pub/Sub connection, a service account, etc.)

My IoT devices are completely unaware of k8s, they don't have enough resources to have any knowledge of k8s. I'm using STM32 MCU with only 128KB of RAM. I have written a Rust embedded runtime which contains only a small WASM interpreter and network layer via CoAP protocol. That is already taking almost all of the RAM. The devices are meant as source of data to the edge k8s cluster.

you might be able to do the composition you need using minibroker; it's an OSB compatible broker that runs Helm scripts against the k8s cluster it's installed in.

It's reassuring that we are having the same ideas. Unfortunately minibroker also doesn't support ARM yet https://github.com/kubernetes-sigs/minibroker/issues/120 (If you see ARM issues recently, there are probably from me 😄). Also so far I've been using only Rust in my thesis and I'm just starting to get comfortable with it. So instead of learning Go, I've resorted to writing a Rust OSB myself. I created a library for that https://github.com/jiayihu/brokerapi-rs, which I'll improve as I get a better real-world use-case for the lib.

I'd really love being able to run Helm scripts like minibroker do, but I don't think I can do that for now since Helm it's written in Go. I'm falling back to creating the resource templates dynamically in Rust, template by template, for now. For the scope of my thesis is sufficient. But in future, if there's any way, I'd love to support Helm.

I already also tested that both WASM images and Helm charts can be packaged and pushed/pulled from the new Google Artifacts Registry.

I'd be happy to take a PR. Cross-compiling in go is pretty straightforward, it looks like arm64 would work, but not wasm:

Thanks for suggesting how to do it! I already had good hopes since service-catalog do supports ARM. The devil lies in the details though, I'm using Alpine Linux which has musl libc instead of glibc. I already had issues even with Rust, but I hope everything goes smoothly. I'll definitely try what you suggested and open a PR if it works. (Compiling in WASM is definitely not needed and I would strongly recommend using Rust for that in case).

Thank you very much for the help you shared!