Fix for #358 - Githubissues

AtlasOfLivingAustralia / biocache-store

Occurrence processing, indexing and batch processing

Other

7 stars 24 forks source link

Fix for #358 #360

Closed vjrj closed 4 years ago

vjrj commented 4 years ago

A simple fix for #358.

Using this I can end a process-local-node without performance degradation of 32M records.

biocache-store-1 2019-12-29 00:33:46,948 INFO : [Cassandra3PersistenceManager] - All threads have completed paging
biocache-store-1 2019-12-29 00:33:47,102 INFO : [ProcessLocalRecords] - Total records processed : 32027457 in 85081.65 seconds (or 236.33792 minutes) readCount=32027457 updateCount=32027457 updateFailCount=0

PS: I send previously this over master.

ansell commented 4 years ago

Thanks, the travis build for this patch is now working against develop so I will merge it in

nickdos commented 4 years ago

@djtfmartin could you QA this (acceptance testing)?

vjrj commented 4 years ago

Just for the record, Manash confirmed me the past week that this patch solved their issues.