Support for Linux containers

mikeminutillo commented 11 months ago

We are currently working on enabling ServiceControl to run in Linux containers.

The work is broken down into distinct steps. As each step is completed, a new version of ServiceControl will be released and this issue will be updated.

Based on the remaining work we are 90% confident that we will be able to produce Linux container images for all types of ServiceControl instances by the end of June 2024.

This estimate will be revised each time that the issue is updated.

Subscribe to notifications on this issue to stay up to date.

This issue is locked to ensure that notifications are sent only for progress updates. Please use the Particular Discussion Group for any questions, comments, or concerns.

Overview

✅ Attempted in-place upgrade to RavenDB 5
✅ Split Audit capability into dedicated instance type
✅ Added option to run instances without ingestion
✅ Spike different persistence mechanisms
✅ Introduce persistence seam for ServiceControl Audit instances
✅ Add RavenDB 5 persistence seam implementation for ServiceControl Audit instances
✅ Introduce partial persistence seam for ServiceControl Error instances
✅ Make full-text indexing optional
✅ Switch from Windows-specific code to Microsoft Generic Host
✅ Created Windows docker container images for all ServiceControl instances
✅ Enabled selecting transport and persistence options at runtime
✅ Upgrade persistence for Audit and Error instances to RavenDB 5
✅ Produce versions of ServiceContol that run on .NET
🚧 Produce Linux Docker images (in progress)

Each step requires the previous one to be completed before it can be started.

✅ Upgrade persistence for Audit and Error instances to RavenDB 5 (complete)

✅ ServiceControl Audit instances (completed)
✅ ServiceControl Monitoring instances (N/A — do not persist data)
✅ ServiceControl Error instances (completed)

ServiceControl stores data in an embedded RavenDB instance. Older versions of RavenDB require .NET Framework and use ESENT (a Windows component) for storage, which means they can't run on linux. RavenDB 5 targets .NET and uses Voron for storage), which means it can run on Linux.

The change from ESENT to Voron makes it impossible to perform an in-place upgrade. We are introducing a persistence abstraction to allow us to choose an implementation at runtime.

graph TD
  ServiceControl --> abstraction(Abstraction)
  abstraction -..-> raven35(RavenDB 3.5 implementation)
  abstraction -..-> raven5(RavenDB 5 implementation)

ServiceControl Audit instances can already use RavenDB 5. All new instances target RavenDB 5 and we have instructions for migrating existing instances from RavenDB 3.5 to RavenDB 5.

✅ Produce versions of ServiceContol that run on .NET

We are using the terms ".NET" and ".NET Framework" as described in ".NET vs. .NET Framework for server apps".

Parts of ServiceControl still require .NET Framework, which is not available on Linux. To run on Linux, all of ServiceControl must run on .NET.

Going forward, we will migrate ServiceControl from .NET Framework to .NET. Among other things, this will require us to switch from from ASP.NET Web API to ASP.NET Core and switch to a version of SignalR which runs on .NET (which requires a corresponding change to ServicePulse).

🚧 Produce Linux Docker images (in progress)

There are still a few "features" of ServiceControl which absolutely require Windows:

Running as a Windows service
Configuration using the Windows Registry
Sending critical errors to the Windows Event Log

We will modify these features in the .NET version of ServiceControl, providing appropriate alternatives for Linux environments, and publish Docker images for each type of ServiceControl instance.

DavidBoike commented 7 months ago

With the release of ServiceControl 5.0, one of the most significant blockers to supporting Linux containers (the use of RavenDB 3.5, which depends on .NET Framework) has been removed, bringing us much closer to the ultimate goal. Additionally, work on the next step, updating ServiceControl to run on modern .NET, is already underway.

ServiceControl 5.0 uses RavenDB 5 for both Audit and Error instances, provides a new installer experience, and provides general performance, stability, and logging improvements. See the release notes for more details.

Details for how to upgrade to the new version can be found in the upgrade guide.

We have updated the description of this issue to reflect the current state of our progress. Based on the remaining work, we are 80% confident that we will be able to produce Linux container images for all types of ServiceControl instances in 4 to 6 months, which works out to somewhere during Q2 of 2024.

mauroservienti commented 2 months ago

We would like to share an update on our progress on producing versions of ServiceControl that run on .NET.

The ServiceControl master branch (not yet released) now runs on .NET 8. The codebase has been migrated to use ASP.NET Core 8, ASP.NET Core SignalR 8, and NServiceBus 9.

We discovered that the changes in SignalR are more complex than expected, so it is taking longer than anticipated to guarantee that ServicePulse and ServiceInsight remain compatible with the new version of ServiceControl.

The remaining work to produce versions of ServiceContol that run on .NET is:

Ensure ServicePulse is compatible with ServiceControl running on .NET (as well as older versions running on .NET Framework)
Complete the testing phase and fix any issues blocking the release

After that, work will begin immediately to produce Linux Docker images, which we expect to be much less complex than the previous steps.

We're 80% confident we can produce Linux container images for all types of ServiceControl instances by the end of June 2024. We'll provide another update when we've produced versions of ServiceControl that run on .NET.

mauroservienti commented 2 months ago

With the release of ServiceControl 5.2, all the blockers to supporting Linux containers (the dependency on .NET Framework and classic ASP.NET) have been removed.

Details for how to upgrade to the new version can be found in the upgrade guide.

The work to produce Linux container images has already started. We are 90% confident that we will be able to provide Linux container images for all types of ServiceControl instances by the end of June 2024. We'll provide another update when we have more information about the progress of the container images work.

DavidBoike commented 2 weeks ago

While we aren't 100% done with all the documentation, we know you're all looking forward to having ServiceControl on Linux containers. So today, we're doing a soft release so you can test things out.

ServiceControl uses 4 different container images:

particular/servicecontrol - the Error instance
particular/servicecontrol-audit - the Audit instance
particular/servicecontrol-monitoring - the Monitoring instance
particular/servicecontrol-ravendb - the database used by the Error and Audit instances
- Note that it's highly recommended that the Error and Audit instances both have independent database containers.

We will be working on shaping up the official documentation over the next couple weeks, but we think the documentation for each image on Docker Hub should give you enough to play with if you just can't wait. We'll continue to post updates on this thread as work on the documentation progresses.

We've also created a dedicated thread on our discussion site for ServiceControl on Linux containers early access questions. We'll be monitoring that thread carefully for questions and feedback. We'd love to hear from you there. There's still the opportunity to incorporate feedback into additional releases during the month of July.

We know you've been waiting for this for a long time. Take it for a spin and let us know what you think.

Particular / ServiceControl