search

WallarooLabs / wally

Distributed Stream Processing
https://www.wallaroolabs.com
Apache License 2.0
1.48k stars 68 forks source link

Feature: Official Wallaroo AMI for running Python apps #2333

Open pzel opened 5 years ago

pzel commented 5 years ago

As a Wallaroo user I'd like to leverage tools that rely on AMIs to deploy my app So I can iterate/go to market faster and spend less time reinventing the wheel

Based on recent experience, l feel that having a publicly-available Wallaroo AMI would greatly ease deploying and managing users' Wallaroo+Python apps in the (Amazon) cloud.

Tangentially related is https://github.com/WallarooLabs/wallaroo/issues/2310.

enilsen16 commented 5 years ago

A heroku add-on could also be a great way to add creditability.

ref: https://devcenter.heroku.com/articles/becoming-an-add-on-partner

SeanTAllen commented 5 years ago

You don't have to do "As a" pzel.

SeanTAllen commented 5 years ago

@enilsen16 until someone asks for a Heroku add-on I would be reticent to add one when we could spend that time working on Kubernetes integration or other deployment options that we know Wallaroo users are using.

pzel commented 5 years ago

@SeanTAllen unless you really dislike the "agile" flavor of "As a", I'd like to keep using it because it helps me empathize with the user and their use case. We can also rephrase the issue to prose, like: "The issue at hand is that we'd like to help users set up Wallaroo clusters in the AWS cloud in as few steps as possible (...)".

I appreciate the "as a" pattern for its brevity -- in cases where it does apply. I'm not fanatical about the concept ;)

@enilsen16 If my knowledge is up-to-date, Heroku doesn't support inter-node (inter-dyno) clustering, so it's unclear what kind of gymnastics we'd need to do to set up a multi-worker cluster. (Phoenix has a PubSub overlay via Redis exactly for the Heroku use-case, because standard Erlang clustering doesn't apply)

SeanTAllen commented 5 years ago

@pzel I dislike the brevity of the as pattern. There's a lot of nuance and important details that can get lost in it.

enilsen16 commented 5 years ago

Heroku doesn't support inter-node (inter-dyno) clustering

@pzel True, but if I am reading the documentation correctly. When you create an add-on you can use any service (Not heroku) to host it.

ref: https://devcenter.heroku.com/articles/how-add-ons-work