Fleeting plugin for Hetzner #121

Closed perlun closed 8 months ago

[UPDATE 2024-02-19] A repo for a Hetzner GitLab fleeting plugin has now been created and some initial work has been done: https://gitlab.com/fleeting-plugin-hetzner/fleeting-plugin-hetzner

Hi,

Now, the issue is just that GitLab is moving to a new autoscaling mechanism, which is completely understandable given that Docker Machine is deprecated upstream and they (GitLab) are maintaining their own fork of it. The status on the new GitLab-provided plugins are as follows:

AWS: Beta, [1]
GCP: Experiments, [2]
Azure: Experiments, [3]

The above fleeting providers and the "fleeting" abstraction layer project are available here: https://gitlab.com/gitlab-org/fleeting.

"Someone" should write a fleeting provider for Hetzner, so we can continue using our nice Hetzner cloud provider with GitLab even in the future. There is no "panic" in this sense; this page mentions "FY26 Q2 (MAY-JUNE 2025)" for "supporting the GitLab maintained Docker Machine fork for autoscaling runner on VM's on the significant public providers". Note that "significant public providers" regretfully does not include Hetzner in this case, I believe, so there's still a risk of breakage at an earlier stage. This plugin is well-maintained (thank you, @JonasProgrammer :pray:) so it's possible that potential issues could be resolved in the plugin in that case.

Regardless, it's time for us, as a community around this plugin, to think about the future. We need a fleeting plugin for Hetzner.

I wouldn't mind doing this myself, but I haven't really done any Go programming before and my time (for projects like this) is limited. Do we have any other people who would like to volunteer? It's possible we can do it as a bit of a "community effort", but there still needs to be at least 1-2 (preferably 2-3) core developers involved to do the majority of the work, realistically.

In other words, if you have been longing to learn Go and/or the Hetzner cloud API, now is a golden chance to show some :heart: back to the community to which I think we are all rather grateful (or if not, should be! We are truly standing on the shoulders of giants. :bow:)

Before I conclude, I want to take the time to thank all people who has been involved in this project, and particularly @JonasProgrammer once more. Your efforts have been invaluable and I personally appreciate them a lot. You rock! :rocket: :smiley:

More details

Hi,

first of all, thank you for the kind words. Given that this started out just as a pet project to learn some Go and screw around the the Hetzner Cloud beta, it is impressive to look back and see how far this has come.

Regarding the GitLab Fleeting part, I would be happy to lend a hand with my little Hetzner Cloud experience, if it could be of any help. But frankly, I do not think I am qualified to act either as a lead or even core developer of the mentioned efforts, mostly due to the fact, that I have no experience in running and developing for GitLab. The closest I came was running GL Community from then Helm chart once for a few days for evaluation purposes. Aside from that, there is currently some medical issues to deal with causing longer delays in responding (exhibit 1: this issue) than I would like to be able to provide.

Nonetheless, when there is anything I can do to support this effort, be it in working out some hcloud quirks, providing resources for interop/migration from docker-machine-driver-hetzner or joint ventures in presenting interests to the upstream hcloud(-go) team, I would be really happy to do that.

Again, thank you for the laurels, though they are kind of unjustified, given the amount of work people aside from me have put into this. I really do hope someone will pick this up and go forward with it. Anything I can, limited by the circumstances at the time, do to support this effort, I will do.

Cheers!

Hi @JonasProgrammer, sorry to hear about the medical issues. Hope you get better soon. :pray:

I actually discussed this a bit with a colleague and I started looking into it after all. :slightly_smiling_face: The project is far from being generally usable yet, but we have made some initial progress. You, or everyone else for that matter, are welcome to follow the progress here if you are interested: https://gitlab.com/fleeting-plugin-hetzner/fleeting-plugin-hetzner

We are a long-term GitLab user in our org, and I can clearly understand your point about this being a bit hard to achieve (and also motivate yourself in doing) if you aren't really using GitLab yourself. (Would be interested to hear about your non-GitLab use case for this yourself, or have you never actually used the plugin for anything apart from experimentation perhaps? Either way, it's something that has been ended up being useful to us and certainly others too, so again, thanks for that. :heart:)

There's also an upstream GitLab issue that mentions our plugin which was created after I mentioned it to Darren Eastman at their end, https://gitlab.com/gitlab-org/gitlab-runner/-/issues/37370. We are looking to get some sponsoring from them, initially for more CI minutes (GitLab.com has stricter limits for this than GitHub), but hopefully the whole project can eventually be brought into the https://gitlab.com/gitlab-org/fleeting or similar namespace)

The first prerelease version of the plugin has now been created: https://gitlab.com/fleeting-plugin-hetzner/fleeting-plugin-hetzner/-/releases/v0.1.0. We are also testing this internally.

I'll close this issue for now, since the work has essentially been started. :rocket:

JonasProgrammer / docker-machine-driver-hetzner

Fleeting plugin for Hetzner #121

More details