When doing writes to solr, add retry controls with backoff

lucidworks / spark-solr

Tools for reading data from Solr as a Spark RDD and indexing objects from Spark into Solr using SolrJ.

Apache License 2.0

446 stars 250 forks source link

When doing writes to solr, add retry controls with backoff #315

Open nddipiazza opened 3 years ago

nddipiazza commented 3 years ago

When indexing to solr, it is very common to have the solr server hiccup while you are using spark-solr to bulk import huge amounts of data. Need the ability to control retries with detailed retry logging.

It should also support backoff algorithm as well to give the server a chance to recover.