[Backport 2.x][FEATURE] Support batch ingestion in TextEmbeddingProcessor & SparseE…

[chishui]

…ncodingProcessor (#744)

• Support batch ingestion in TextEmbeddingProcess & SparseEncodingProcessor

Signed-off-by: Liyun Xiu xiliyun@amazon.com

• Update Changelog

Signed-off-by: Liyun Xiu xiliyun@amazon.com

• Add UT and IT

Signed-off-by: Liyun Xiu xiliyun@amazon.com

• Add comments

Signed-off-by: Liyun Xiu xiliyun@amazon.com

• Sort texts by length before sending for inference

Signed-off-by: Liyun Xiu xiliyun@amazon.com

• Make consistent check for inferenceList

Signed-off-by: Liyun Xiu xiliyun@amazon.com

---

Signed-off-by: Liyun Xiu xiliyun@amazon.com (cherry picked from commit afd1215a930a14cb889ea8751997a8a18cce5d1a)

Description

[Describe what this change achieves]

Issues Resolved

[List any issues this PR will resolve]

Check List

• [x] New functionality includes testing.
  • [x] All tests pass
• [x] New functionality has been documented.
  • [x] New functionality has javadoc added
• [x] Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.