feat: optional token separators for search index schema

adilansari commented 1 year ago

What type of PR is this? (check all applicable)

[ ] Refactor
[x] Feature
[ ] Bug Fix
[ ] Optimization
[ ] Documentation Update

Description

Optionally specify tokens to split the texts, texts will be tokenized by space, new-line and specified characters.
```
@TigrisSearchIndex(ORDERS_INDEX_NAME, { tokenSeparators: ["/"] })
export class Order {
// fields
}
```

Related Tickets & Documents

Related Issue #
Closes #

Added/updated tests?

[x] Yes
[ ] No, and this is why: please replace this line with details on why tests have not been included
[ ] I need help with writing tests

Is this change backwards compatible?

[x] Yes
[ ] No, and this is why: please replace this line with details on why?

Does it require updates to Tigris docs?

[ ] Yes, and here is the link: please create an issue in tigris-docs repo and replace this text as tigrisdata/tigris-docs#123
[ ] No

Checklist

[x] npm run build - builds successfully
[x] npm run test - tests passing
[x] npm run lint - no lint errors

[optional] Are there any post deployment tasks we need to perform?

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.03 :tada:

Comparison is base (a527ec4) 90.37% compared to head (7695f59) 90.41%.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #382 +/- ## ========================================== + Coverage 90.37% 90.41% +0.03% ========================================== Files 29 29 Lines 5644 5667 +23 Branches 662 663 +1 ========================================== + Hits 5101 5124 +23 Misses 542 542 Partials 1 1 ``` | [Impacted Files](https://app.codecov.io/gh/tigrisdata/tigris-client-ts/pull/382?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tigrisdata) | Coverage Δ | | |---|---|---| | [src/decorators/tigris-search-index.ts](https://app.codecov.io/gh/tigrisdata/tigris-client-ts/pull/382?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tigrisdata#diff-c3JjL2RlY29yYXRvcnMvdGlncmlzLXNlYXJjaC1pbmRleC50cw==) | `100.00% <100.00%> (ø)` | | | [src/schema/decorated-schema-processor.ts](https://app.codecov.io/gh/tigrisdata/tigris-client-ts/pull/382?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tigrisdata#diff-c3JjL3NjaGVtYS9kZWNvcmF0ZWQtc2NoZW1hLXByb2Nlc3Nvci50cw==) | `100.00% <100.00%> (ø)` | | | [src/search/search.ts](https://app.codecov.io/gh/tigrisdata/tigris-client-ts/pull/382?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tigrisdata#diff-c3JjL3NlYXJjaC9zZWFyY2gudHM=) | `97.65% <100.00%> (+0.05%)` | :arrow_up: | | [src/types.ts](https://app.codecov.io/gh/tigrisdata/tigris-client-ts/pull/382?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tigrisdata#diff-c3JjL3R5cGVzLnRz) | `91.45% <100.00%> (+0.04%)` | :arrow_up: | | [src/utility.ts](https://app.codecov.io/gh/tigrisdata/tigris-client-ts/pull/382?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=tigrisdata#diff-c3JjL3V0aWxpdHkudHM=) | `92.95% <100.00%> (+0.11%)` | :arrow_up: |

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

github-actions[bot] commented 1 year ago

:tada: This PR is included in version 1.3.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket:

tigrisdata-archive / tigris-client-ts