search

habitat-sh / habitat

Modern applications with built-in automation
https://www.habitat.sh
Apache License 2.0
2.61k stars 316 forks source link

Investigate desirability of running multiple instances of a package on a single supervisor #5095

Open christophermaier opened 6 years ago

christophermaier commented 6 years ago

Some deployments may need to run multiple instances of a package on a single supervisor / machine. (This applies to bare-metal Supervisors, and situations where Docker is not being used).

Due to how the Supervisor's filesystem is set up (see #4975), about the only practical way to achieve this right now is to have a separate package (and thus service name) for each instance of the service you want to run. This is unnecessarily profligate, and could potentially be addressed by allowing users to specify a service name override at service load time, effectively decoupling a package from a service.

adamhjk commented 6 years ago

What's the actual use case?

I get the theoretical one - has anyone ever had a real issue with this? The benefit of not allowing multiple instances of a given service is pretty high, imho.

Atalanta commented 6 years ago

We have an actual use case for this right now. Chris and I just got off a call with one of our teams, which led to this issue.

Probably non-trivial to describe the exact use case in writing, and also potentially sensitive, but happy to jump on a call with you to run you through the thinking - Chris and I may have missed an elegant solution.

gggiroux commented 6 years ago
christophermaier commented 6 years ago

@adamhjk Basically what @Atalanta and @gggiroux said... we've got a workaround for how the Supervisor currently behaves, but I thought it might be good to hash out whether or not we would ultimately want the Supervisor to support something like this natively.

rsertelon commented 6 years ago

Would the services run with the same group in the identified use cases? If not then the fs could include it?

bdangit commented 6 years ago

This would be really nice feature for the :bear:metal folks - just like you said @christophermaier. We have a service where we would apply a certain runtime configuration and then we would like to stand up that same service but with a different runtime configuration.

adamhjk commented 6 years ago

On the exact same piece of physical hardware, without virtualization or containerization?

If I put my brain in the way-back machine, I remember running things like dev/qa farms with services on different ports to squeeze more efficiency. Is that what we're talking about? To me, this adds a bunch of complexity and possible footguns if you aren't careful.

If we wanted to make this change, then yes, the way you would do it os to have each service start writing itself to /hab/svc/{service}.{group} rather than just {service}.

Atalanta commented 6 years ago

Yes, multiple instances of the same service, sometimes a slightly different version of the service, running on the same (very powerful) hardware, without virtualisation/containerisation to isolate it.

adamhjk commented 6 years ago

I'm pretty sure we should just do what y'all want. The pedant in me struggles, though, because it's so much more repeatable and safe to isolate it at the system level. Pretty sure that doesn't override your need. 👍

bdangit commented 6 years ago

I would say it’s not that I want to run one supervisor with many services under it. I want each service to have its own supervisor. And to a certain point that @atalanta is saying, we would be running our services on powerful machines. No virtualization or containerization.

adamhjk commented 6 years ago

@bdangit you can ahve that today

adamhjk commented 6 years ago

--override-name and set the listen ports to different ports.

christophermaier commented 6 years ago

@adamhjk There are some pretty significant downsides to using --override-name right now; see #4975

rsertelon commented 4 years ago

Adding a comment on this one. I'm currently facing the issue on my own machine (personal server).

I have a service that simply serves a given directory with a better index.

In my opinion, it does make sense to run the service multiple times, once for each root, on different port/ip address (https://github.com/rsertelon/rsertelon-plans/blob/master/h5ai/default.toml).

It's been a long time this issue has been opened, is there something that can be done? How could I help if so?

Another solution would be to create multiple packages, but I'd rather not do that if possible.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.

rsertelon commented 2 years ago

I would still be interested in a solution to that issue personally.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.