Adjust settings for better bulk indexing

iDigBio / idb-backend

iDigBio server and backend code for data ingestion, media processing, record indexing, and data API.

GNU General Public License v3.0

7 stars 0 forks source link

Closed danstoner closed 3 years ago

danstoner commented 3 years ago

change elasticsearch.helpers.streaming_bulk options to stop raising a fatal / exit when there is a single record exception during bulk indexing.
adjust number of replicas, etc. when indexing begins, set back to config values after bulk indexing is completed.