elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
14.18k stars 3.5k forks source link

Logstash and Elastic Cloud on Kubernetes Announcement #15030

Closed flexitrev closed 4 months ago

flexitrev commented 1 year ago

Overview

We are announcing the Technical Preview of Logstash on Elastic Cloud on Kubernetes (ECK). Enabling users to deploy and manage Logstash instances on Kubernetes clusters. Taking advantage of Kubernetes’ powerful scaling and management capabilities, users can handle large volumes of data while simplifying the complexities of managing a distributed system. Using Kubernetes, users can more easily scale up and down their Logstash deployments based on processing needs. This ensures your data processing pipeline can handle large volumes of data without having wasted capacity.

The initial release includes everything users need to deploy Logstash instances on Kubernetes including documentation, stack monitoring, pipeline definition and support for references to Elasticsearch instances. This functionality is currently in Technical Preview, as we test and add features. We are pleased to make the deployment of Logstash on Kubernetes available under the Basic license which is Free and open to all customers. Advanced features, such as auto-scaling, may require an enterprise license in the future.

Supported Functionality

We have added a Custom Resource Definition (CRD) for Logstash to ECK, supporting the necessary parameters to deploy Logstash on Kubernetes clusters. The Logstash operator supports the deployment, configuration and management of Logstash 8.6+.

Included Features:

Limitations

By making it available to users as a Technical Preview, we hope to provide the community with much desired functionality and solicit feedback on feature development. A technical preview means functionality can change without deprecation warnings. Be sure to pay close attention to ECK release notes.

The following features are under consideration, but will not be available in the initial release.

Where can I get it?

We plan on releasing with the next Elastic stack release, in late May. The logstash operator, updated CRD and user facing documentation will be available as part of the Elastic Cloud on Kubernetes for anyone to download. We want to hear back from the community about their experience deploying and managing Logstash with Kubenetes, please use comments below, or file issues against the cloud-on-k8s repository.

asazallesmilner commented 1 year ago

How does this interact with the licensing of resources managed by the operator in enterprise clusters and how that effects how many ERU's we need.

flexitrev commented 1 year ago

How does this interact with the licensing of resources managed by the operator in enterprise clusters and how that effects how many ERU's we need.

The current Elastic subscription agreement specifically excludes Logstash from billable enterprise software, which means Logstash deployments are not counted towards enterprise resource units

flexitrev commented 4 months ago

How does this interact with the licensing of resources managed by the operator in enterprise clusters and how that effects how many ERU's we need.

The current Elastic subscription agreement specifically excludes Logstash from billable enterprise software, which means Logstash deployments are not counted towards enterprise resource units

The Elastic subscription agreement has been updated to distinguish between ECK and all other deployments. This is to be consistent with the ECK pricing strategy. Under the new agreement, ECK managed Logstash resources are not excluded from Billable software. The Opex savings made available by the Logstash operator, through the ease of deployment, management and scaling more than offset the cost of ERUs associated with ECK managed Logstash resources.