.bldr.toml should allow users to auto-promote to channels other than unstable

[elliott-davis]

@tashimi commented on Sat Dec 16 2017

This is expected/documented behavior but apparently doesn't exist in the code base.

Additional channel promotion

-You can specify a list of additional channels that packages should automatically publish to. By default, software is only published to the unstable channel.

-toml -# .bldr.toml -[hab-launcher] -channels = ["stable"] -

---

@tashimi commented on Sat Dec 16 2017

• add behavior
• re-add docs removed in #4327

Aha! Link: https://chef.aha.io/features/APPDL-67

[smartb-pair]

@tashimi / @elliott-davis, as of today this has become production-critical for us (had a minor production outage because of the "Polka Problem" described here, which we think can be circumvented by auto-promoting our origin's packages to a staging channel or similar and then setting HAB_BLDR_CHANNEL to staging in our Builder secrets.

cc: @bixu

[smartb-pair]

(It seems at the moment that our only alternative is to stop using Builder entirely and retain only the artifact storage (Depot) functionality.)

[raskchanky]

I did some research yesterday into what it would take to enable this feature and it doesn't look that bad. It looks like it was at least partially implemented at some point in the past, because the .bldr.toml file has support in it for channels. @chefsalim mentioned that he removed parts of this feature at one point, but I'm not clear what the reasons were for that, or if there's something preventing us from putting it back together.

[chefsalim]

The feature was removed as it was broken and needed re-design - the promote then was happening in the job scheduler, which is not where we want actual job tasks to happen. Also the bldr.toml could override the sandboxing the way the original implementation worked, which is not what we want.

So from a requirements perspective, the decisions to be made are around:

• Does autopromotion happen as builds complete, or does it happen only after the full build group has built successfully (the original implementation did the latter)
• Does auto-promote happen if any of the packages fails to build?
• Is the channel specifier static or can it be dynamically specified in the bldr.toml?
• Can multiple channels to be specified?
• Is 'stable' one of the channels that can be specified?
• Should there also be a test hook that gets run before auto-promote?
• (Future) this should fit in with exposing builder events, so people can build new workflow that's exactly what they want.

[rsertelon]

My 2cts here:

• autopromote should happen when the whole group is built correctly (at least in the origin that triggered the build)
• I'd say no, the risk here is that we'd risk inconsistent deps for hab users
• static would be enough IMHO, for now at least. provided we can easily change while being retro-compatible
• I'd say single channel for now, provided we could change this later while being retro-compatible
• stable has a strong meaning, I think it would be best not to allow it, this should be done explicitly IMHO
• that would be super cool, first thing could be to run builds with DO_CHECK=1, maybe as an option for autopromote? As for other things, it might depend on @eeyun and @nellshamrell work on tests for core plans
• +1

[raskchanky]

My thoughts:

• Auto-promote should only happen when the whole group has completed successfully
• Auto-promotion doesn't happen if any of the packages fail to build
• Channels should be specified in .bldr.toml. The feedback I'm hearing from users here revolves around wanting more control over their build process, and letting them specify the channels they want in the .bldr.toml would give a lot of control. I don't think it would be that much extra work to make this happen, and the benefits are huge.
• I think multiple channels are fine. The difference in effort required to implement this change with multiple channels vs a single channel is very small, and the potential benefits here for some users is likely very big. This feels like a big win for little effort.
• I don't see a problem with letting stable be one of the channels, particularly if we require that the entire group build successfully before auto-promotion happens. What other criteria are there for promoting something to stable other than a successful build?
• A test hook that gets run before auto-promotion would be really nice, but to me, this feels like a nice-to-have vs a must-have (unless the effort to implement this is small, then why not).
• Definitely

[rsertelon]

@raskchanky re stable being one of the channels. I think technically, it works if all jobs succeed indeed. I was thinking core-plans and I think there we should forbid that (at least for now?), but that could be done in a travis check maybe.

[raskchanky]

@rsertelon That's a fair point, and we might also be able to include that as a configuration option inside the .bldr.toml file. Something like promotion_to_stable_allowed = true, so it could be tweaked as needed.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.

[stale[bot]]

This issue has been automatically closed after being stale for 400 days. We still value your input and contribution. Please re-open the issue if desired and leave a comment with details.