Replace Ofelia by Chadburn for dynamic job management

[Crazyphil]

Summary

Replace Ofelia by its fork Chadburn, which support dynamic configuration management. This avoids having to restart the Ofelia container when new jobs are added or existing ones are modified, which would potentially interrupt running jobs or spam the log files when a container goes down.

Motivation

Mailcow currently facilitates Ofelia for scheduling jobs. When using Docker labels for jobs, only one Ofelia container must be present on a given Docker host, because otherwise, jobs will be scheduled multiple times. This, however, is no problem, because Mailcow clearly is open to extending the core functionality beyond a pure mailserver, e.g. by adding websites or installing additional software. Reusing the single scheduler container for other projects on the same host therefore should not be an issue.

Unfortunately, development on Ofelia has pretty much stalled, which lead to forking it as Chadburn. This new project allows dynamically (re-)loading job configuration from Docker labels without having to restart the Ofelia container. Dynamic configuration management is a very useful feature that is also used e.g. by Traefik Proxy and decouples containers from the infrastructure other containers provide. Adding different Docker projects to a Docker host while using the Mailcow Ofelia scheduler would be possible without interrupting any mailcow service, even when these projects go up and down frequently.

Additional context

Switching to Chadburn would be very easy. It is published to Docker Hub and supports the same configuration labels. The only difference is the use of the chadburn prefix instead of ofelia.

[andryyy]

Thanks for the great FR.

We should implement it in staging for testing.

[mkuron]

Please verify whether Chadburn fixes the memory leak that Ofelia had for a long time (#4272, https://github.com/mcuadros/ofelia/issues/157).

[qchn]

I am new to mailcow and I really enjoy using it but while researching why my setup needs so much memory ressources, I found out it's because of Ofelia and so I stumbled upon this Feature Request which I can highly support.

[maietta]

Please verify whether Chadburn fixes the memory leak that Ofelia had for a long time (#4272, mcuadros/ofelia#157).

Hi there. I am the maintainer of Chadburn and discovered this issue thread while browsing for critical updates or fixes that should be implemented in Ofelia and in turn, likely to be incorporated in my hard fork. I agree, memory leaks should not be something people should deal with. I am still learning Go, so I am not sure if I can solve this one yet, but I will give it a shot.

I'll see what I can do to address the memory leak issue over the weekend and into next week.

[andryyy]

Amazing. Thank you!

[maietta]

I forked then swapped out Ofelia with the latest version of Chadburn on an 8GB ram instance of Ubuntu 20.10. While it's only been about an hour or so, the system has never spiked above 3GB of ram on my "bugkillerz.com" test email testing VPS.

I will keep this running for a while to see how this performs. I have alerts in place for if ram goes above 3.5GB. Since I am not familiar with the resource utilization expected of a minimal install of Mailcow, I don't have any reference to know if this is good or not.

I suspect if the docs say to use a 6GB + 1GB swap as minimum, then 3GB isn't too bad.

[maietta]

I made the mistake of bringing in all the changes over the last 3 months into Chadburn from Ofelia only to realize that Ofelia never performed the most important pull request that caused me to fork Chadburn in the first place.

So v1.0.1 of Chadburn no longer seems to support the detection of changes in Docker containers. I will be spending my day tomorrow, Christmas trying to fix this.

[maietta]

Update: Mailserver running smoothly since I last posted here, with no memory leak using Chadburn in place of Ofelia.

I'm still planning on a version 2.x in the coming months but I just wanted to let ya'll know that things are smooth on my end.

[maietta]

Bump. Just letting you know the project is still not dead :)

I plan a full rewrite at some point soon but there are a lot of people using Chadburn and the issues aren't piling up and it's been a long while.

If Ofelia continues to serve this project well, great, but it doesn't appear that much has been going on with it. I want to let you know that Chadburn remains a critical component for my projects so it will always remain maintained, even if not frequently.

[DerLinkman]

We will test and create a replacement pr and integrate it i think in nightly at best @FreddleSpl0it?