support pre-process using NLP from plugin

toluaina / pgsync

Postgres to Elasticsearch/OpenSearch sync

https://pgsync.com

MIT License

1.16k stars 180 forks source link

support pre-process using NLP from plugin #441

Open virtualaidev opened 1 year ago

virtualaidev commented 1 year ago

PGSync version: 2.5.0

Postgres version: debezium/postgres:15

Elasticsearch version: docker.elastic.co/elasticsearch/elasticsearch:8.7.0

Redis version: 6.2.6

Python version: 3.7

Problem Description:

I like to pre-process documents through NLP for text embedding before indexing to ES. Can I use the plugin for this? I believe plugin works for each document not the list of documents, or am I wrong with this?

toluaina commented 1 year ago

HI and sorry about the delay, Yes the plug-in operates on each document and allows you to perform custom transformations