Open TreyRhodes1 opened 2 years ago
I was able to resolve this issue by updating line 177 of src/main/scala/com/lucidworks/spark/util/SolrSupport.scala with this bit of code
new HttpSolrClient.Builder()
.withBaseSolrUrl(shardUrl)
.withHttpClient(getCachedCloudClient(zkHost).getHttpClient)
.withConnectionTimeout(240000) # new
.withSocketTimeout(600000) # new
.build()
Is there a way to set this configurations without modifying the source code?
we are also facing same issue while updating records on a huge collection. is there any workaround rather than changing the code
We recently upgraded from the spark-solr jar 3.2.3 to 4.0.0 and upgraded our spark version to 3.1.2. Our application now runs utilizing Kubernetes instead of Yarn. Our workloads complete successfully most of the time. When they do fail it's because our application has failed all retries to query Solr. This is the full error:
Once this error has showed up enough times for one query, it causes the entire application to fail.
This error only seems to happen when our Solr queries take longer than a threshold, appearing to be 120000ms. We've determined that Zookeeper and Solr do nothing to sever the connection between Solr and our client application, so I suspect a timeout is set somewhere in our application or dependent jars, specifically spark-solr. Is there a configuration set somewhere to timeout Solr queries that take too long?