search

opensearch-project / opensearch-spark

Spark Accelerator framework ; It enables secondary indices to remote data stores.
Apache License 2.0
12 stars 18 forks source link

OpenSearch index shard based parallel processing #396

Open penghuo opened 4 days ago

penghuo commented 4 days ago

A partition of an OpenSearchTable is backed by an OpenSearch Index. Each partition is split into a configurable number of shards, which are then distributed across the cluster.

We support users directly accessing OpenSearch indices without creating a table, as described in . https://github.com/opensearch-project/opensearch-spark/pull/399.