Closed reset closed 6 years ago
Ongoing notes:
Do not create a separate plan file, is my strong feeling.
@adamhjk Not that I necessarily disagree, but could you elaborate on your reasons?
My preference right now is the same file as well but I hadn't thought about it long enough to really double down on that opinion. The concerns that I have so far are what happens if there's both a plan.sh
and composite-plan.sh
file and the changes to any tooling that we create which will think of plan.sh
as main()
but now we may have two main()
s
I'm not sure that I fully understand the API / API proxy scenario, but I've been trying to conceive how something like this might work for a rails application that has application sever processes and delayed_job processes. I keep thinking I'd want it to work just the way everything does today except that the run hook sees a proc file and starts processes based on what's in there
I don't see why this should be in the same file. The plan.sh is already complecting build instructions with runtime behavior. This would be adding another responsibility overloading it even more.
In nomad the are distinct concepts of job, group and task. A single service would correspond to a task and a composite plan would be a group. In kubernetes the pod is also a distinct concept that aggregated multiple sub containers.
My point is since other systems consider a group of things to be something different than an element in the group I'm not sure if this approach is right.
Using the composite pattern which treats a plan or an aggregation of multiple sub plans identically from the pov of collaborators might make sense but even so I'm not sure if the definition of the aggregation belongs in the Dame file.
/brain dump out
On Aug 10, 2017 8:32 PM, "Jamie Winsor" notifications@github.com wrote:
My preference right now is the same file as well but I hadn't thought about it long enough to really double down on that opinion. The concerns that I have so far are what happens if there's both a plan.sh and composite-plan.sh file and the changes to any tooling that we create which will think of plan.sh as main() but now we may have two main()s
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/habitat-sh/habitat/issues/2902#issuecomment-321636109, or mute the thread https://github.com/notifications/unsubscribe-auth/AII1MEw33iAr0PgtWNfZWACkTnMa-aFIks5sW0y0gaJpZM4Oyz86 .
Right now, we have a 1:1 mapping between a given package and a service it can run.
What we want to be able to say is that we want a 1:N mapping between a packages and services it can run?
Lets say I have a package adam/francisco
, which has two services - francisco-api
and francisco-db
. When I say:
hab start adam/francisco
Is the expected behavior here that I would get two services, one named adam/francisco-api
and another named adam/francisco-db
? Or would you need to launch them separately?
Another way to look at this is that you want to have a completely different animal, akin to a procfile. All it would do is list a set of packages you want run, and let you get the UX behavior I say above. In that case, you would have:
adam/francisco-core
adam/francisco-api
-> adam/francisco-core
adam/francisco-db
-> adam/francisco-core
adam/francisco
-> composite(adam/francisco-api adam/francisco-db)
Are we in the same spot? Which is it?
@adamhjk My understanding is that it's more like the first scenario you describe. When you load your adam/francisco
"package", that's largely saying "load up adam/francisco-api
and adam/francisco-db
and run both of those services on the same machine". You interact with them in terms of the top-level adam/francisco
"service", as opposed to manipulating the API and DB services individually.
In this example api & db feel like two separate unrelated things? I haven't had any experiences yet where I've wanted this. I think I only want this sort of behavior when the services are all running off the same codebase. With a rails app and delayed_job it doesn't make much sense to have two packages because it's all the same executables, just different startup params. I think the reality of needing to copy all the app scripts & config files and get them set up in /hab/svc
really ?highlights? this
For this instead of a composite plan I'd be happy with parameterized run hooks and maybe some more smarts in init....
hab start adam/francisco-api --run web
hab start adam/fancisco-api --run jobs
ls /hab/svc/francisco-api/hooks
init
run -> web/run
web\run
jobs\run
This would still leave you the capability to run different pieces on different servers.
Yep I think @chrisortman is right. I was halfway through drafting a similar post. The pain is much more apparent for that sidecar pattern or the rails foreman pattern.
We don't really have an issue with wrapping things up, or with starting different things and getting them to talk to one another. What we DO have pain with is when your app has a dependency on itself or on a sidecar service that functions in support of your running thing. This is why php users will see this a lot.
The examples I've run into recently are with the nomad package and with some work I did with one of our community members in atlanta. Nomad for example - the binary runs in multiple varied states but theres only a single binary. In one case the nomad binary can run in agent mode, but doing so means that dockerd must be running on the underlying system. It's trivial to add a line to my runhook that starts and backgrounds dockerd, but then I lose control over that process.
With the folks in atlanta they were simply trying to package up a very lightweigt webapp with nginx + node to serve static content. The challenge they ran into was pretty similar.
I also find myself wondering if focusing on the way we read/handle run-hooks is the appropriate path. For example: if I could have run-dockerd
, and run-nomad
or run_1
and run_2
and the supervisor could start and watch each process correctly without a need for some extra layer on the plan that might be suitable enough.
Or another idea: a clean way for the user to specify in the run hook itself: start this other package first. or start this package in this separate context first before executing the rest of this run hook
I have the use case described most-closely by @chrisortman: a single Rails codebase with both web and jobs services. The only difference between the services is the run hook. What's shared between the two services includes configuration: their templates and computed final configs should be identical.
It seems like there are a couple patterns at play here and I'm sure there are even more in the world and all are very valid. A couple things that come to my mind here:
We can currently declare runtime and build time deps. What if we added a svc_deps
that would be similar to runtime deps but in addition to grabbing the dep, basically wrapped a hab svc start
around that package. I'd imagine you would end up with a tiny "meta" package that would typically just have a name, origin and svc_deps.
For cases where you would want multiple services that allm run the same binary but with different args. Would it suffice to have a single plan for building that binary and then different plans that tyook a runtime dep on the binary and used it with its own specific args in the run hook
@mwrock I think I'm doing something like what you are suggesting in your 2nd bullet right now, but not with a binary
https://github.com/ui-icts/sparc-request/tree/master/habitat
I think my primary gripe about the set up is that when the dependent packages need / get restarted doesn't feel deterministic. I think it works most of the time for me by coincidence, but one of my first troubleshooting steps is to stop/start the web & jobs packages
@eeyun this feature is exactly that, a way to represent the sidecar pattern 😄
Thats what I figured @reset. I guess from my perspective just the DB example is a little misleading if only because lots of times having a DB connection is trivial to orchestrate and they aren't usually to hard to manage with the current habitat behaviors. Those paired services with web-server+static content, or service + supporting service are harder to compose currently.
User Story
As a plan author, I need a way to represent a service which is made up of one or more processes, so I can interact with a collection of services in the same Supervisor as one
Background
We currently have one kind of plan which is represented by a
plan.sh
or aplan.ps1
. A plan outputs a package which, when installed, can be started by a Supervisor to start a service. It can be said that a plan/package is a 1:1 mapping to a service.Some services require another to run and we need a way to represent this "sidecar" pattern. An example of this is an API server and an API proxy.
Changes to Plan-Build
We need a way to represent that the plan is a composite plan. We could do this in a few different ways such as:
pkg_composite = true
plan.sh
such ascomposite-plan.sh
We also do not need to allow any additional artifacts aside from the
plan.sh
in a composite plan. There's no need for the additional artifacts - read on for details.Promise Mapping
Once we've indicated that we're building a composite plan we also need a way to configure some mappings for any promises that the package exposes (aka,
exports
/binds
). We also need a way to note that an export of a service of the composite plan satisfies a bind of another. For example,builder-api
needs to satisfy the bind forbuilder-api-proxy
.We need to verify that there are no collisions in the key space of exports or binds. If one package in the composite exports
hello
and a second also does, we will need to fail the build (and should fail before any work is done)We will want to record this in a new METADATA file indicating:
Changes to Hab CLI
All operations with the CLI should be atomic across all service members of the composite package/service. For example: installing a composite package should install all packages it points to. Another way to think about a composite package is as a "pointer package" (sometimes referred to in other systems as a virtual package).
We may want to consider representing a composite service in the status output of running services as something like:
All
hab pkg
andhab svc
commands should be refreshed to support these changesConfig Apply
Since we want a composite service to be addressable as a service group itself, we will also need to make some alterations to
hab config apply
.Currently we run
hab config apply <SERVICE_GROUP> <VERSION_NUMER> [file]
which takes a toml file representing a single service's configuration. We should allow the toml file to be split into multiple sections, one for each service in the composite service:Changes to Butterfly / Supervisor
With all of the above changes we may also need to add a new rumor type,
CompositeServiceRumor
, which contains the metadata for promise mappings.The config apply changes described above may also result in the need for us to generate a new type of rumor for
CompositeServiceConfigRumor
. This would contain the combined configuration. If we decide to go this route, we will need to make sure that we reject config changes to the individual service groups which make up the composite service group - contains a combinedLastly, we should not generate a service directory at
/hab/svc/{name}
for a composite service.