search

snowplow / snowbridge

For replicating streams across clouds, accounts and regions
Other
15 stars 7 forks source link

Add arm64 support #302

Closed vbhamidipati closed 6 months ago

vbhamidipati commented 10 months ago

We are hoping to run everything on AWS using Graviton instances but noticed that Snowbridge does not have binaries/docker images that support arm64.

Are there any existing plans/efforts to support Snowbridge on arm64? Happy to submit a PR as well.

colmsnowplow commented 10 months ago

Not something we've considered, and it's unlikely to become a priority for us to be totally honest - we've found typical EC2/ECS/K8s instances works well for our purposes so far, so we're unlikely to put time to it.

If you'd like to have a go at building your own image, you can see how it's done in the makefile.

Then if you need a docker image, you can see how we produce that in the Dockerfiles at the root of the repo.

Out of curiosity - is there a specific reason for wanting to use Graviton? What are the benefits you're after, over other types of instances?

vbhamidipati commented 10 months ago

Thanks for the pointer.

As for why Graviton, it is mostly for cost and performance reasons. Our organization is able to get a better deal from AWS when we are able to use more of the same type with Reserved instances. Stream-collector and Enrich already have ARM images. Makes our design simpler and cost-effective if we are able to run all on a larger pool of similar instance types.

colmsnowplow commented 10 months ago

Ah, that makes lots of sense, thanks for explaining!

Apologies for my brevity in the last message, in the busy end of year I think I was more blunt than I meant to be - I meant to convey that I'm sceptical about our finding time to schedule this in the shorter term, given what I can see on our agenda at the moment. However I should've mentioned that that's not my call, I've raised the request internally with our product team.

I don't think we'd schedule in a release for just this, but next time we need to do a release perhaps it can find its way in. :)

colmsnowplow commented 10 months ago

I should have mentioned - if you were inclined to make a PR we'd be happy to add it to the next release, but not able to give a timeline on when that might be. :)

vbhamidipati commented 10 months ago

Hi @colmsnowplow, I have create a PR into the repo with the changes I believe are needed. I tried to stay within the boundaries of how the current ci/cd is structured. LMK what you think: https://github.com/snowplow/snowbridge/pull/303