[Feature Request] Include SQL and PPL as default high-level languages for OpenSearch and OpenSearch Dashboards

[anirudha]

Is your feature request related to a problem? Please describe

SQL and PPL (Piped Processing Language) will be integrated as primary query languages in OpenSearch. These high-level query languages will now be available by default in the minimal distribution of OpenSearch, significantly enhancing the usability and accessibility for all users.

OpenSearch, known for its robust search and analytics capabilities, has seen an increasing demand for high-level query languages. SQL, with its widespread use and familiarity, along with PPL's powerful and flexible exploration and investigation capabilities, provide users with intuitive and efficient ways to query and manipulate data. This integration ensures that users can leverage these powerful tools immediately upon installation, streamlining operations and reducing setup complexity.

"By integrating SQL and PPL directly into OpenSearch, we are making it easier for our customers to access and utilize advanced query capabilities right out of the box,"

"This enhancement underscores our commitment to providing powerful, user-friendly tools that meet the evolving needs of our customers."

The integration of SQL and PPL into the minimal distribution of OpenSearch will be managed as modular components, ensuring that performance remains optimal and the core codebase remains efficient.

Features

1. Enhanced Usability

   • Built-in support for SQL and PPL, making it easier to perform complex queries and data manipulation.
   • Immediate access to powerful query tools without additional installations.

2. Improved Accessibility

   • Lower entry barrier for users familiar with SQL and PPL, facilitating a smoother transition and more efficient use of OpenSearch.
   • Broadens the appeal of OpenSearch to a wider audience.

3. Streamlined Operations for partners, developers and users

   • Reduces setup complexity and maintenance overhead by integrating SQL and PPL into the minimal distribution. Enable plugin developers with APIs and sdks to start developing in sql/ppl.

Implementation details

• Modular Integration: SQL and PPL will be included as modular components( plugins), ensuring they do not bloat the core codebase while ensuring ease of distribution.

• Extensive Testing: with over 2.5k Comprehensive tests sql codebase will ensure stability and performance.

• Comprehensive Documentation: Detailed reference user guides and tutorials to help users get started with SQL and PPL in OpenSearch.

The integration of SQL and PPL as primary languages in OpenSearch marks a significant enhancement in usability and accessibility. By addressing the growing demand for high-level query languages, we are ensuring that OpenSearch remains a powerful, user-friendly solution for all users.

Describe the solution you'd like

include a build system change only/ no code movement.

Related component

Build

Describe alternatives you've considered

No response

Additional context

No response

[dblock]

Aren't SQL and PPL languages already included in the default distribution of OpenSearch, downloaded from https://opensearch.org/downloads.html?

The -min distribution exists for backwards compatibility with open distro / Elasticsearch. We discussed not distributing it altogether but there was a lot of community pushback.