How to avoid overwrite of records

lucidworks / spark-solr

Tools for reading data from Solr as a Spark RDD and indexing objects from Spark into Solr using SolrJ.

Apache License 2.0

446 stars 250 forks source link

How to avoid overwrite of records #361

Open 4d1in3 opened 1 year ago

4d1in3 commented 1 year ago

Currently, I'm trying to write millions of records into solr and I want to avoid overwrites. data is coming via kafka and spark structured streaming job is writing.

UI shows an option to avoid overwriting, is there any option in the lib to replicate this behavior